-- Logs begin at Tue 2022-08-09 01:14:10 UTC, end at Tue 2022-08-09 02:58:02 UTC. -- Aug 09 02:26:39.369432 dv-cinder-852461-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 02:26:39.382122 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 02:26:39 2022] *** Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: nodename: dv-cinder-852461-1-iscsi Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: machine: x86_64 Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: clock source: unix Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: pcre jit disabled Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: detected number of CPU cores: 4 Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: current working directory: / Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: detected binary path: /usr/bin/uwsgi-core Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: your processes number limit is 47680 Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: your memory page size is 4096 bytes Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: detected max file descriptor number: 2048 Aug 09 02:26:39.384303 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: lock engine: pthread robust mutexes Aug 09 02:26:39.385182 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: thunder lock: enabled Aug 09 02:26:39.385182 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 02:26:39.385182 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 02:26:39.423542 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: Python main interpreter initialized at 0x56513e0ad9c0 Aug 09 02:26:39.423542 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: python threads support enabled Aug 09 02:26:39.423542 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: your server socket listen backlog is limited to 100 connections Aug 09 02:26:39.423542 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: your mercy for graceful operations on workers is 90 seconds Aug 09 02:26:39.423891 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 02:26:39.423956 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: *** Operational MODE: preforking *** Aug 09 02:26:39.424252 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: *** uWSGI is running in multiple interpreter mode *** Aug 09 02:26:39.424252 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: spawned uWSGI master process (pid: 80483) Aug 09 02:26:39.424448 dv-cinder-852461-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 02:26:39.425138 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: spawned uWSGI worker 1 (pid: 80496, cores: 1) Aug 09 02:26:39.425339 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: spawned uWSGI worker 2 (pid: 80497, cores: 1) Aug 09 02:26:39.425513 dv-cinder-852461-1-iscsi devstack@keystone.service[80483]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 02:26:44.933184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 02:26:44.933184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: from cryptography.utils import int_from_bytes Aug 09 02:26:44.936092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 02:26:44.936092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: from cryptography.utils import int_from_bytes Aug 09 02:26:44.940786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 02:26:44.940786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: from cryptography.utils import int_from_bytes Aug 09 02:26:44.941362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 02:26:44.941362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: from cryptography.utils import int_from_bytes Aug 09 02:26:44.983107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 02:26:44.983289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 02:26:44.983475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] command line args: [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 02:26:44.983781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 02:26:44.983959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ================================================================================ {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 02:26:44.984275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] admin_token = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.984572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] config_dir = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.984797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.984967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] config_source = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.985190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] debug = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.985398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.986816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.986816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.986816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.986816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.987113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.987113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_config_append = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.987302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.987534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_dir = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.987715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_file = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.987908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 02:26:44.988055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.988231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 02:26:44.988411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.988585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] command line args: [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 02:26:44.988763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.988928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 02:26:44.988928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ================================================================================ {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 02:26:44.989238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.989447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] admin_token = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.989595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.989595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.989944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] config_dir = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.989944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.990289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.990564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] config_source = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.990844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] debug = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.991989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_config_append = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.992791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_dir = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_file = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.993890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] publish_errors = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.994893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] standard_threads = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.995964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] use_journal = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] use_json = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.996694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] use_stderr = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] use_syslog = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] publish_errors = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.997934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.997934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.998168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.998284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.998408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.998529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] standard_threads = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.998529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.998744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.998866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.998993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.999108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.999108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] use_journal = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.999325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.999448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] use_json = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.999448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] use_stderr = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.999448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] use_syslog = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:44.999779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:44.999893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 02:26:45.000007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.000958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.001924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.002858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.003988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.004929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.005973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.006818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.007986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.008988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.009932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.010897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.011941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.012990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.013938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.014931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.015967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.016929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.017945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.018892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.019921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.020938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.021947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.external = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.022936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.password = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] auth.token = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.023904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.024884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.025970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.external = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.026934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.password = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] auth.token = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.027888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028691 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.028932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.029957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.030971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.031916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.032968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.033917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.034919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.035962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.036866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.037976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.038974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.039916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.040979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.041911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.042900 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.043954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.044891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.045944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.046919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.047983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.048913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.049829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.050997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.051918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.052927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.053996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054516 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.054990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] role.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] role.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.055921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.056863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.057988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.058936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] role.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] role.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.059973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.060902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.061957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.062952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.063992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064805 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.064920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.065983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066457 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.066815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.connection = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.067958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.068919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.069953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.070904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.071941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.connection = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.072305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.073235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 02:26:45.074200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 02:26:45.097965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.098212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.098438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.101165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.101396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.101625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.108235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.111459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.136537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.136771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.137783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.138027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.188017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.188365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.238120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.238383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.238753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.238986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.302494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.302665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.302747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.302826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.302935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.303694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.326501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.326690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.326787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.326883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.326972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.327974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.338937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.339175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.352972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.353107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.353184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.353323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.353407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.353540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 02:26:45.371492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.371835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 02:26:45.371961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.372154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 02:26:45.372405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.372532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.372668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 02:26:45.372763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 02:26:45.373035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.373162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.373310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.373573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.373699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.373805 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.373979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.374287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.386124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.386447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.386573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.386698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.386794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.387003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.387144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.387311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.393865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.394103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.396979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.397209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.397335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.397459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.397554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.397761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.397889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.398872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.399945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.423052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.423839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.424574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.424866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.425367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.425653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.429467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.429756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.430044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.430301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.430589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.430875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.436637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.436961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.437436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.437721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.441062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.441342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.441635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.441762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.441931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.442972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.443081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.443173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.443404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.443722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.459016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.459297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.459422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.459640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.463303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.463630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.466479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.466780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.466905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.467093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.471509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.471862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.474733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.475119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.477913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.478209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.478334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.478527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.478655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.478827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.478955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.479059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.479141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.479372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.479500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.479665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.495281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.495573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.495699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.495826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.495921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.496114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.496243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.496323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.502989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.503203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.504556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.504791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.504965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.505060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.507696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.507929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.508075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.508188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.508279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.508525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.508653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.508808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.508895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.509080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.520190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.520427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.520557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.520696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.520790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.521918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.522119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.538400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.538778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.541357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.541738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.544323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.544692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.545899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.546287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.547440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.547724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.547850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.547992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.548078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.548309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.554461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.554766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.556068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.556369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.559022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.559288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.559416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.559557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.559642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.559866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.559994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.560148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.567900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.568189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.568317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.568449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.568617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.568786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.575512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.575641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 02:26:45.575842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.575988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.576901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.577001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.577162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.577294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.577513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.577642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.577855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.578936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.579066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.579204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.579292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 02:26:45.612790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56513e0ad9c0 pid: 80496 (default app) Aug 09 02:26:45.612935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56513e0ad9c0 pid: 80497 (default app) Aug 09 02:26:45.616071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b3560-4e26-44d5-90aa-1bc255de44bb None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:45.616355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b3560-4e26-44d5-90aa-1bc255de44bb None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:45.616620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b3560-4e26-44d5-90aa-1bc255de44bb None None] PATH_INFO: `/v3` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:45.620986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Aug 9 02:26:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:26:53.297596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:53.298272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:53.298272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:53.304748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:26:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:26:53.311808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:53.312253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:53.312376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:53.390058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: INFO dbcounter [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] Registered counter for database keystone Aug 09 02:26:53.410725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG oslo_db.sqlalchemy.engines [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80496) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 02:26:53.422934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writer thread running {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 02:26:53.605599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG passlib.handlers.bcrypt [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80496) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 02:26:53.606005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG passlib.handlers.bcrypt [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80496) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 02:26:53.661010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:26:53.662671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:26:53.796843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:26:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 02:26:53.972375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:54.048766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: INFO dbcounter [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] Registered counter for database keystone Aug 09 02:26:54.073675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG oslo_db.sqlalchemy.engines [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80497) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 02:26:54.084558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writer thread running {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 02:26:54.203711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82606826-3c23-4ce7-a7b3-968776a74136 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:54.209177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:54.209907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:54.210196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:54.210481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:54.218932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:54.269812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING keystone.common.rbac_enforcer.enforcer [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 02:26:54.270104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924771bf-8d70-43ae-b312-4c7816c5fb8b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:54.280194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Aug 9 02:26:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:26:54.290226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:54.313532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db988fdc-c86e-48cf-b2f2-4be44547c678 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:54.320892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:54.322028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:54.322334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:54.322604 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] PATH_INFO: `/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:54.324274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] RBAC: Authorizing `identity:get_project(project_id=aaeb91952f2c4c9f8cf8e3a10b5518c9)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:54.373978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING keystone.common.rbac_enforcer.enforcer [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 02:26:54.374332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa4ec3-ea22-4a2a-8b15-77911552887f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:54.378981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Aug 9 02:26:54 2022] GET /identity/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:26:56.334462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5445-fad3-42fc-88b9-4a29774abab2 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:56.335115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5445-fad3-42fc-88b9-4a29774abab2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:56.335115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5445-fad3-42fc-88b9-4a29774abab2 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:56.339264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:26:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:26:56.345919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:56.347060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:56.347214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:56.377069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:26:56.378633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:26:56.388527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:26:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 02:26:56.410539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5445-fad3-42fc-88b9-4a29774abab2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:56.425920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5445-fad3-42fc-88b9-4a29774abab2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:56.431555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8953139-57e7-4ef0-9041-caa8f18d7c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:56.432161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8953139-57e7-4ef0-9041-caa8f18d7c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:56.432438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8953139-57e7-4ef0-9041-caa8f18d7c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:56.432712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8953139-57e7-4ef0-9041-caa8f18d7c54 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:56.440667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8953139-57e7-4ef0-9041-caa8f18d7c54 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:56.442476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8953139-57e7-4ef0-9041-caa8f18d7c54 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:56.451220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Aug 9 02:26:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:26:56.456819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:56.470051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87de9d-8fb4-4c01-90f1-fbd42f73a062 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:56.480412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eddbe6-36b4-4cfd-bec9-c43ac798784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:56.481066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eddbe6-36b4-4cfd-bec9-c43ac798784c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:56.481350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eddbe6-36b4-4cfd-bec9-c43ac798784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:56.481623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eddbe6-36b4-4cfd-bec9-c43ac798784c admin admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:56.483067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eddbe6-36b4-4cfd-bec9-c43ac798784c admin admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:56.485646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eddbe6-36b4-4cfd-bec9-c43ac798784c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:56.488152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Aug 9 02:26:56 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:26:58.532952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd7829-6bac-4c3c-bb8f-0ad8d327f6ef None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.533507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd7829-6bac-4c3c-bb8f-0ad8d327f6ef None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.533588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd7829-6bac-4c3c-bb8f-0ad8d327f6ef None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.534958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699416e7-5fcb-47d2-9147-c2a51b50facd None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.535366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:26:58.535466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699416e7-5fcb-47d2-9147-c2a51b50facd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.535566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699416e7-5fcb-47d2-9147-c2a51b50facd None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.536857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:26:58.540356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.541356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.541766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.542708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.543133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.543540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.565292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:26:58.566515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:26:58.589483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG passlib.handlers.bcrypt [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80497) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 02:26:58.589905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG passlib.handlers.bcrypt [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80497) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 02:26:58.635271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:26:58.636882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:26:58.648011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 7/12] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:26:58.670726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:58.695415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:58.737395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 6/13] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:26:58.759014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:58.777367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4808b6-7987-4cc3-a009-3485c82324ad None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:58.812802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:58.813446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.813728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.814002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.825158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:26:58.834003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:26:58.835177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:26:58.836413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 8/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:26:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:26:58.841284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:58.862134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3148b37-95af-4b2d-8526-dc04bd41face None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:58.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:58.899116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:58.899984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.900267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.900537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.924982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:58.946277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed690aa5-8592-4e36-ab98-bb2ef7f737c9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:58.969005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:58.970580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:58.982275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:58.982889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:58.983173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:58.983442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:58.984118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:26:58.984412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:58.985796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:58.996911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 9/15] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:26:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:26:59.002553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:59.071287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 7/16] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 02:26:59.077409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:59.081382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:59.181122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1a9c8-f5da-4347-ac06-5196f815d003 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:59.213395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc6e8e-5a8c-4fb3-9eb2-4a546c8800f9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:59.214677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:59.215580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:59.215855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:59.216154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:59.234898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:59.236505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:59.246255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:59.246887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:59.247164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:59.247432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:59.248565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:59.250000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:59.251607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 10/17] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:26:59 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:26:59.254802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:26:59.255790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:26:59.256671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:26:59.256671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:26:59.256671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:26:59.256671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 8/18] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:26:59.257219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:59.261795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:59.272774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:59.315119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:59.356073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589e0a9-2903-4b36-b542-77ba4826c0b4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:59.361751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441567ca-c83f-4f8a-963f-2b2423610050 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:59.389048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:59.389697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:59.389990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:59.390377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:59.391432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:59.392871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:59.394482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 11/19] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:26:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:26:59.394736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:59.395353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:59.395632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:59.395903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:59.396552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:26:59.396855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:59.398318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5082cf-f6e1-421f-bbc9-35ab52de9f26 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:59.400413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:26:59.418570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 9/20] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:26:59.486972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:26:59.507375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f52b-eac1-4a90-aecc-fd9528f1b74e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:26:59.543702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:26:59.544414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:26:59.544630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:26:59.544903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:26:59.545646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.domain.id=default, user.id=b1cb94a6532045fc84cecdb848b352fb) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:26:59.545960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:26:59.547493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2d98e-fc64-4848-9e15-2a125b2224f0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:26:59.555416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 12/21] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Aug 9 02:26:59 2022] GET /identity/v3/role_assignments?user.id=b1cb94a6532045fc84cecdb848b352fb&scope.domain.id=default&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:01.804879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70297ad-7729-4270-8a8a-8169c380804f None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:01.805372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70297ad-7729-4270-8a8a-8169c380804f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:01.805372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70297ad-7729-4270-8a8a-8169c380804f None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:01.806813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 10/22] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:01.812325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:01.813142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:01.813623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:01.833355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:01.834977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:01.915334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 13/23] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:01.941083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70297ad-7729-4270-8a8a-8169c380804f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.059599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70297ad-7729-4270-8a8a-8169c380804f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:02.080244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:02.102011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2940ef9-de53-4141-9b32-3c43c5b33f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:02.102687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2940ef9-de53-4141-9b32-3c43c5b33f25 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.102965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2940ef9-de53-4141-9b32-3c43c5b33f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.103234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2940ef9-de53-4141-9b32-3c43c5b33f25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.118174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2940ef9-de53-4141-9b32-3c43c5b33f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f2940ef9-de53-4141-9b32-3c43c5b33f25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:02.124913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:02.128813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:02.131235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 11/24] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:02.136431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.196821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bac32-1149-47bb-b5a2-a5ee5aaceb99 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.197151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bac32-1149-47bb-b5a2-a5ee5aaceb99 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.197359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bac32-1149-47bb-b5a2-a5ee5aaceb99 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.198952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 12/25] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:27:02.204707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49626115-400e-4648-9433-51b0af6916b6 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.205048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49626115-400e-4648-9433-51b0af6916b6 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.205253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49626115-400e-4648-9433-51b0af6916b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.235222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-49626115-400e-4648-9433-51b0af6916b6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:02.236776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-49626115-400e-4648-9433-51b0af6916b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:02.238526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:02.265754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc7a50-9ea2-4da2-993b-873cbd465f73 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:02.300669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:02.301306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.301583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.301855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.306176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:02.306176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:02.306176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135150f-9b51-48c0-995e-25caef633eb5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:02.313389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 14/26] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:02.314395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eeced-ee9f-4237-a4ce-d0d905a2e99a None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.314679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eeced-ee9f-4237-a4ce-d0d905a2e99a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.314953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eeced-ee9f-4237-a4ce-d0d905a2e99a None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.316156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 15/27] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:02.318812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eeced-ee9f-4237-a4ce-d0d905a2e99a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.392719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 13/28] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:02.393859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0a824-2fd6-4e06-8875-3817fa0d9518 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.394223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0a824-2fd6-4e06-8875-3817fa0d9518 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.394504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0a824-2fd6-4e06-8875-3817fa0d9518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.432943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eeced-ee9f-4237-a4ce-d0d905a2e99a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:02.472943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:02.473606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.473886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.474185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.475470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:02.476952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:02.478939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 16/29] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:02 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:02.479554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.483917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:02.548892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-37d0a824-2fd6-4e06-8875-3817fa0d9518 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:02.550248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-37d0a824-2fd6-4e06-8875-3817fa0d9518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:02.589569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5798b-dc06-4544-b5c4-0b37c5bafdef None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:02.621887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 14/30] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:02.623126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192607e9-5996-4fdc-8f5b-7e712d5832b8 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.623419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192607e9-5996-4fdc-8f5b-7e712d5832b8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.623692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192607e9-5996-4fdc-8f5b-7e712d5832b8 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.625091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 15/31] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:02.625680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e594c1-3e16-4ad9-96ec-5f78cc9d20f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:02.625813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192607e9-5996-4fdc-8f5b-7e712d5832b8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.626414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e594c1-3e16-4ad9-96ec-5f78cc9d20f0 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.626694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e594c1-3e16-4ad9-96ec-5f78cc9d20f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.626969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e594c1-3e16-4ad9-96ec-5f78cc9d20f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.627676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e594c1-3e16-4ad9-96ec-5f78cc9d20f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:02.629246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e594c1-3e16-4ad9-96ec-5f78cc9d20f0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:02.663288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:02.731221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192607e9-5996-4fdc-8f5b-7e712d5832b8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:02.765538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:02.766213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.766487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.766761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.768115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:02.769582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:02.770871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 17/32] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 9 02:27:02 2022] POST /identity/v3/roles => generated 219 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:02.771524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 16/33] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:02.772374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.772547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.772809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.773270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.794891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:02.796757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:02.866894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:02.876397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 18/34] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:02.877242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:02.886884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016013a9-ce66-45c7-a761-421f3af7650d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:02.890517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:02.926575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:02.927567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:02.927810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:02.928090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:02.940435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:02.942060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:27:02.958841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:02.960080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:27:02.961084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:27:02.961084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:27:02.961084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:02.961084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 17/35] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:02.961084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:03.003596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd709-e63e-4afd-b46f-f44a8dac2d61 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.041906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef6e59-ab17-435d-b19c-8c6e08c203c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.042912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef6e59-ab17-435d-b19c-8c6e08c203c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.043341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef6e59-ab17-435d-b19c-8c6e08c203c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.043755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef6e59-ab17-435d-b19c-8c6e08c203c6 None admin] PATH_INFO: `/v3/domains/default/users/b1cb94a6532045fc84cecdb848b352fb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.046029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef6e59-ab17-435d-b19c-8c6e08c203c6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b1cb94a6532045fc84cecdb848b352fb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.047886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef6e59-ab17-435d-b19c-8c6e08c203c6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.067084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 19/36] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Aug 9 02:27:02 2022] PUT /identity/v3/domains/default/users/b1cb94a6532045fc84cecdb848b352fb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:03.068569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac1f5-739e-4f92-a55f-11d99021bc69 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.069127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac1f5-739e-4f92-a55f-11d99021bc69 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.070143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac1f5-739e-4f92-a55f-11d99021bc69 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.071683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 20/37] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:03.072617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac1f5-739e-4f92-a55f-11d99021bc69 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:03.082645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cd657-43f1-488c-a27b-1497892d47c3 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.132544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.133119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.133405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.133836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.134681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.136456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.148883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:03.206505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac1f5-739e-4f92-a55f-11d99021bc69 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.251814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.252478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.252809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.253140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.253876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:03.254210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.255732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadd774-e247-4fe1-a3be-06234e2d9b01 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.275743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 21/38] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:03.277050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b024b-be9c-4cb1-b6c5-95b28973ba4e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.277342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b024b-be9c-4cb1-b6c5-95b28973ba4e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.277622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b024b-be9c-4cb1-b6c5-95b28973ba4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.289460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 18/39] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 9 02:27:02 2022] POST /identity/v3/roles => generated 225 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:03.290789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:03.307047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3d4b024b-be9c-4cb1-b6c5-95b28973ba4e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:03.308292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3d4b024b-be9c-4cb1-b6c5-95b28973ba4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:03.313493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:03.410147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34b0e4-a690-4362-8ab4-098e652254ea None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.449665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.450719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.450825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 22/40] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:03.451646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.452131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.452316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c590-e2e2-417c-b28b-9fa8e1301ed5 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.452612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c590-e2e2-417c-b28b-9fa8e1301ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.452895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c590-e2e2-417c-b28b-9fa8e1301ed5 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.453600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.454333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 23/41] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:03.455754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.460382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.460676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.460962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.483436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:03.484720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:03.502431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 19/42] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:27:03 2022] POST /identity/v3/projects => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:03.503748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:03.549095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:03.592779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 24/43] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:03.606559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:03.619185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:03.652754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0adef-278a-48cb-bd95-aa3186b6dce3 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.694293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.695086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.695369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.696009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.696714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.698705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.721404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 20/44] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 9 02:27:03 2022] POST /identity/v3/roles => generated 223 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:03.749629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e98b04-4b12-4e89-adcd-7b055c473c96 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.788119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1405e1d-b5ec-4894-9dc7-4a444e9c9fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.788847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405e1d-b5ec-4894-9dc7-4a444e9c9fbf None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.789136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405e1d-b5ec-4894-9dc7-4a444e9c9fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.789411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405e1d-b5ec-4894-9dc7-4a444e9c9fbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.790694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1405e1d-b5ec-4894-9dc7-4a444e9c9fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.792210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1405e1d-b5ec-4894-9dc7-4a444e9c9fbf None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.793953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 25/45] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:03.799922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:03.911137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c424b59-c861-447d-bc3d-5d1b69822261 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:03.938628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:03.948126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f66d7-917a-4903-84f2-6b76a17ed18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:03.948704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f66d7-917a-4903-84f2-6b76a17ed18f None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:03.948995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f66d7-917a-4903-84f2-6b76a17ed18f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:03.949279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f66d7-917a-4903-84f2-6b76a17ed18f None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:03.949947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f66d7-917a-4903-84f2-6b76a17ed18f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:03.951758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f66d7-917a-4903-84f2-6b76a17ed18f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:03.981077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 21/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:27:03 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:27:05.196233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e061b-41ec-44b3-89b3-168cdfe93a8b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.196828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e061b-41ec-44b3-89b3-168cdfe93a8b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.196828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e061b-41ec-44b3-89b3-168cdfe93a8b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.198136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 26/47] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:05.203693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.204087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.204286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.224046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:05.225549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:05.294499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 22/48] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:05.317312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e061b-41ec-44b3-89b3-168cdfe93a8b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:05.362916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:05.425772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e061b-41ec-44b3-89b3-168cdfe93a8b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:05.459029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:05.459629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.459912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.460183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.470347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:05.475425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:05.476318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:05.477434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 27/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:05.482580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:05.581688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a65e-efff-440b-84c9-ef787fcad85c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:05.592236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:05.617913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:05.618564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.618837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.619110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.619779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:05.620083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:05.621467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:05.632851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 23/50] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:05.638536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:05.741458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89efd89-ebe7-4280-a304-7df70a0288ec None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:05.742933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:05.776322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:05.776945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.777224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.777492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.778670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:05.780119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:05.781871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 28/51] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:05 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:05.787800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:05.887927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9029b-9f09-4340-aa5a-48be9ae23185 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:05.921465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ee0e7-5b52-49ce-8cea-dd7d96a61c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:05.922100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee0e7-5b52-49ce-8cea-dd7d96a61c6c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.922397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee0e7-5b52-49ce-8cea-dd7d96a61c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.922670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee0e7-5b52-49ce-8cea-dd7d96a61c6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.923793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ee0e7-5b52-49ce-8cea-dd7d96a61c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:05.925260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ee0e7-5b52-49ce-8cea-dd7d96a61c6c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:05.926890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 24/52] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:05.932886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:05.934185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc24fd8-9037-43dc-9b6d-57ec2e3cbb7c None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.934483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc24fd8-9037-43dc-9b6d-57ec2e3cbb7c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.934770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc24fd8-9037-43dc-9b6d-57ec2e3cbb7c None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.936043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 25/53] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:05.941192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:05.941548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:05.942288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:05.960708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:05.962069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:05.995739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:06.030188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 26/54] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:06.033664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fda0c-0d1c-475c-879f-bfa0a3d83c76 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:06.050654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:06.068576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:06.069506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:06.069793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:06.070070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:06.070863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.domain.id=default, user.id=b1cb94a6532045fc84cecdb848b352fb) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:06.071199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:06.072660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:06.078622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 29/55] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Aug 9 02:27:05 2022] GET /identity/v3/role_assignments?user.id=b1cb94a6532045fc84cecdb848b352fb&scope.domain.id=default&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:06.166493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda613a-1a36-42a5-a77e-9b5b4fc95a4f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:06.202667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce806fd-5d74-4046-b39d-36ca1d8f6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:06.203327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce806fd-5d74-4046-b39d-36ca1d8f6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:06.203607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce806fd-5d74-4046-b39d-36ca1d8f6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:06.203884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce806fd-5d74-4046-b39d-36ca1d8f6ee3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:06.205137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce806fd-5d74-4046-b39d-36ca1d8f6ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:06.206674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce806fd-5d74-4046-b39d-36ca1d8f6ee3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:06.208508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 27/56] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:06.214922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:06.257671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:06.323140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc84e1-dca2-4729-b8ce-63ae518e0026 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:06.358071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b736ac-4d02-4b37-bdfa-6a6c148ec084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:06.358704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b736ac-4d02-4b37-bdfa-6a6c148ec084 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:06.358986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b736ac-4d02-4b37-bdfa-6a6c148ec084 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:06.359256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b736ac-4d02-4b37-bdfa-6a6c148ec084 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:06.359898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b736ac-4d02-4b37-bdfa-6a6c148ec084 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:06.361343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b736ac-4d02-4b37-bdfa-6a6c148ec084 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:06.397405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 30/57] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Aug 9 02:27:06 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:08.384142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123137d-5655-45bf-bd1e-ee0d4341d0a8 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:08.384983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123137d-5655-45bf-bd1e-ee0d4341d0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:08.384983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123137d-5655-45bf-bd1e-ee0d4341d0a8 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:08.387220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:08.393415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:08.393868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:08.394002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:08.414248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:08.415476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:08.487316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:08.511796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123137d-5655-45bf-bd1e-ee0d4341d0a8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:08.565614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:08.630683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123137d-5655-45bf-bd1e-ee0d4341d0a8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:08.665820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c23a28-09ee-4a10-9411-7bea28033702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:08.666394 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c23a28-09ee-4a10-9411-7bea28033702 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:08.666670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c23a28-09ee-4a10-9411-7bea28033702 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:08.666944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c23a28-09ee-4a10-9411-7bea28033702 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:08.668112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c23a28-09ee-4a10-9411-7bea28033702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:08.669561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c23a28-09ee-4a10-9411-7bea28033702 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:08.671111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:08.680293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:08.703116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:08.786898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730dc9b-be24-433c-ba88-694e1bdf8214 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:08.822119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177d938-c8c2-4d71-9767-05f061e6438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:08.822748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d938-c8c2-4d71-9767-05f061e6438e None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:08.823028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d938-c8c2-4d71-9767-05f061e6438e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:08.823301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d938-c8c2-4d71-9767-05f061e6438e None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:08.823968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177d938-c8c2-4d71-9767-05f061e6438e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:08.825407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177d938-c8c2-4d71-9767-05f061e6438e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:08.855140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:08 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:11.828419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179e44-9739-41d0-a885-5a658e7c7bd3 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:11.829261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179e44-9739-41d0-a885-5a658e7c7bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:11.829687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179e44-9739-41d0-a885-5a658e7c7bd3 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:11.831274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:11.836539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c30cd-8d5e-4db8-a62a-fa27008692fc None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:11.837426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c30cd-8d5e-4db8-a62a-fa27008692fc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:11.837905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c30cd-8d5e-4db8-a62a-fa27008692fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:11.867179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f81c30cd-8d5e-4db8-a62a-fa27008692fc None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:11.868835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f81c30cd-8d5e-4db8-a62a-fa27008692fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:11.954378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 33/63] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:11.988563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179e44-9739-41d0-a885-5a658e7c7bd3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.066557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043b67-233b-46ea-a9c9-7097274cf0ec None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.067203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043b67-233b-46ea-a9c9-7097274cf0ec None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.067677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043b67-233b-46ea-a9c9-7097274cf0ec None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.069270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 34/64] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:12.075908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd19c2-70cb-4e86-8a92-2cd46eb7f4b5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.076629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd19c2-70cb-4e86-8a92-2cd46eb7f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.078267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd19c2-70cb-4e86-8a92-2cd46eb7f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.087987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.110528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-09cd19c2-70cb-4e86-8a92-2cd46eb7f4b5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:12.111773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-09cd19c2-70cb-4e86-8a92-2cd46eb7f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:12.138300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179e44-9739-41d0-a885-5a658e7c7bd3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.149258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.194182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.194182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.194182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.194182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.220261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:12.225481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:12.226373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:12.227161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 31/65] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:12.232932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.242960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 35/66] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:12.244083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f38db3-62eb-44c8-8f74-fb73471e6adc None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f38db3-62eb-44c8-8f74-fb73471e6adc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.244646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f38db3-62eb-44c8-8f74-fb73471e6adc None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.245907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 36/67] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:12.251180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.251576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.251957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.273726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:12.275163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:12.345318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 37/68] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:12.347491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.375545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9641a-71ac-4d2d-a3e6-4481eba95227 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.423717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.424781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.425275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.425549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.427088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:12.427304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:12.429681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:12.433929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.442200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 32/69] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:12.447557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.476589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.482873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84ea62-1ca4-4932-a198-59f2f24e4c4b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.519799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.520457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.520751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.521028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.531756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:12.536520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:12.537421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:12.538263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 38/70] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:12.543454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.552801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9907a-3440-4b89-9a01-05e01f290d4c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.594288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582025cf-15f3-4022-862f-f739d72de3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.594939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582025cf-15f3-4022-862f-f739d72de3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.595224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582025cf-15f3-4022-862f-f739d72de3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.595741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582025cf-15f3-4022-862f-f739d72de3e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.606336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582025cf-15f3-4022-862f-f739d72de3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-582025cf-15f3-4022-862f-f739d72de3e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:12.611547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:12.612385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:12.613171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 33/71] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:12.614550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63e57-ffcb-479b-b141-1b50f8ca526c None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.614550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63e57-ffcb-479b-b141-1b50f8ca526c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.614550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63e57-ffcb-479b-b141-1b50f8ca526c None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.615874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 34/72] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:12.620443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63e57-ffcb-479b-b141-1b50f8ca526c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.654315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56877d33-7047-4ebd-b9ec-a33dfd5c990c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.676433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.688373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.689011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.689290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.689561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.690726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:12.692188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:12.693769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 39/73] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:12.694560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.709206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.722220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63e57-ffcb-479b-b141-1b50f8ca526c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.756329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.756981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.757244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.757522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.758239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:12.758601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:12.760006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053e227-ad9a-4c1c-869f-ee99fe798d44 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:12.768941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 35/74] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:12.770175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.770465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.770743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.789598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:12.790878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:12.802663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e804-a65a-4d29-9dec-f3853f0d38fc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.836925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.837655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.838037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.838406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.839089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:12.839396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:12.840790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9311645-f774-4cbe-ad5a-4d28513543ab None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:12.849325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 40/75] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:12.850496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54beefb-f44b-4722-9d80-aea2fa7d7744 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.850789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54beefb-f44b-4722-9d80-aea2fa7d7744 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.851069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54beefb-f44b-4722-9d80-aea2fa7d7744 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:12.852277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 41/76] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:12.854267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54beefb-f44b-4722-9d80-aea2fa7d7744 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.863282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 36/77] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:12.864009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:12.907866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.947245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:12.957029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54beefb-f44b-4722-9d80-aea2fa7d7744 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.966859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678803ab-c6ac-48eb-a455-59f342b66f2c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:12.990778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:12.991477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:12.991762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:12.992049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.002438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.003083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.003397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.003638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.018697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.020175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.021816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.022613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.062743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.064223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.118453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.119714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd70ae7-cc83-4ae4-9e2d-3f6be7bd6029 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.153549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a4ed-b59f-4ced-90cf-dab5229fcca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.154198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a4ed-b59f-4ced-90cf-dab5229fcca2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.154522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a4ed-b59f-4ced-90cf-dab5229fcca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.154767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a4ed-b59f-4ced-90cf-dab5229fcca2 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.155892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a4ed-b59f-4ced-90cf-dab5229fcca2 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.157330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a4ed-b59f-4ced-90cf-dab5229fcca2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.158966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 38/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.158966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082293-e92c-40ca-a429-5304351a01da None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.158966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082293-e92c-40ca-a429-5304351a01da None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.158966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082293-e92c-40ca-a429-5304351a01da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.170258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 42/80] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:13.171188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.172751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.179752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-33082293-e92c-40ca-a429-5304351a01da None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:13.180972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-33082293-e92c-40ca-a429-5304351a01da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:13.249874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 39/81] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:13.250753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082293-e92c-40ca-a429-5304351a01da None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.273878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12b69b-726a-449d-8631-0df60d281a88 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.307876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.308519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.308804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.309084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.319062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:13.323974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:13.324880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 43/82] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:13.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.345198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.348226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082293-e92c-40ca-a429-5304351a01da None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.382835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.383408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.383689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.383961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.387373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.436542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216f57-1dca-4ec1-bcd8-403b1a995e40 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.452092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.453580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.470207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.470810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.471179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.471494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.535745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.537237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.550000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 40/83] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:13.551419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.554398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.587064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.638573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 44/84] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:13.641350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.652796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649753c4-30d2-4a46-99e6-abdd4a4c63f3 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.685403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.686024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.686325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.686594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.687754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.689229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.690905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 41/85] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.691672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.741133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca554c0-2d8b-4a2c-8aae-8a4edae116f3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.754713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.775922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.776566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.776878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.777154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.778306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.779769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.781348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 45/86] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.786050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.786130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11d41-300a-4974-959f-a028ce2126b8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.788397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.820163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.820772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.821113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.821384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.822057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:13.822388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.823758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.831648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 42/87] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.832448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.878253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afb6c4-f6bf-49e0-b3f8-0002d81b36b0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.913202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.913766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.914049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.914329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.915546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.917006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.918675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 46/88] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.923718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:13.929053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd153a9d-088b-42d5-b843-391d1de89b22 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:13.962129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:13.962802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:13.963087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:13.963367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:13.964510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:13.965599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:13.965987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:13.967907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:13.968411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.005152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:14.017031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ab2a6-ef34-4305-9d3f-078ba67548c1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.048584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.049198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.049486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.049760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.050530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=b1cb94a6532045fc84cecdb848b352fb) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:14.050840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.052286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.057960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 47/90] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:13 2022] GET /identity/v3/role_assignments?user.id=b1cb94a6532045fc84cecdb848b352fb&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:14.058775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.082179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f2aa4-46c7-4ffc-8469-265dd2e639da None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.128413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.129289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.129582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.129844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.131124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.132728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.167971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 44/91] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:13 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:14.169008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.179110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ea5e-eeaa-4f13-ae75-12edc82dd2f8 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.216863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:14.220929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.222470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.222952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.223453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.225104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.227254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.234310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 48/92] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:14 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:14.235208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.275466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:14.288009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8060d5-dcaf-4aee-9d2e-67a9f247dcdd None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.328075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.328982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.329400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.329810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.330746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=186bbe5bc1074e8890ca95419df7673e) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:14.331056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.332539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.341488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 45/93] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:14 2022] GET /identity/v3/role_assignments?user.id=186bbe5bc1074e8890ca95419df7673e&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:14.342239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.349633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acad8fb-91be-4aa0-9878-4026d265a305 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.407986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.409283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.410224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.410709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.411674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] RBAC: Adding query filter params (role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=186bbe5bc1074e8890ca95419df7673e) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:14.412216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.415328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.428633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 49/94] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:14 2022] GET /identity/v3/role_assignments?user.id=186bbe5bc1074e8890ca95419df7673e&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=14f7282e89144655a81a327a65c198a5 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:14.477264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d252-de3d-4bff-8a2a-7ea14e8302d6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.521001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.521961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.522425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.522836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.556263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:14.610064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.611727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.716188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 46/95] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:14.722513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.773579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:14.834452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ce466-29b0-4088-a007-de89876be9ba None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:14.872799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503c409-5ca8-4781-90e8-561e656027f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:14.873456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503c409-5ca8-4781-90e8-561e656027f5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:14.873740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503c409-5ca8-4781-90e8-561e656027f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:14.874011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503c409-5ca8-4781-90e8-561e656027f5 None admin] PATH_INFO: `/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:14.875247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503c409-5ca8-4781-90e8-561e656027f5 None admin] RBAC: Authorizing `identity:get_project(project_id=c2ac9703a4694dffbf4d792b7f0cf32b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:14.876710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503c409-5ca8-4781-90e8-561e656027f5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:14.878701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 50/96] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:14 2022] GET /identity/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:14.885109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:14.941844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:14.989079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83a485-2d9a-4771-bfd6-708f73ccff95 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:15.023988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:15.024673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:15.024953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:15.025227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:15.026024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=c2ac9703a4694dffbf4d792b7f0cf32b, user.id=186bbe5bc1074e8890ca95419df7673e) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:15.026357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:15.027819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8193-18f0-450f-b050-a2830d8f8e72 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:15.033550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 47/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:14 2022] GET /identity/v3/role_assignments?user.id=186bbe5bc1074e8890ca95419df7673e&scope.project.id=c2ac9703a4694dffbf4d792b7f0cf32b&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:16.668605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298fb51-102d-4b42-8919-40ddfd653de2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:16.669116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298fb51-102d-4b42-8919-40ddfd653de2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:16.669116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298fb51-102d-4b42-8919-40ddfd653de2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:16.670517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 51/98] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:16.676116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:16.676535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:16.676724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:16.698810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:16.700041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:16.769631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 48/99] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:16.811750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298fb51-102d-4b42-8919-40ddfd653de2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:16.936108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298fb51-102d-4b42-8919-40ddfd653de2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:16.950375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:16.979219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda1a55-cf52-4eb9-b71b-9413db2b9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:16.979867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1a55-cf52-4eb9-b71b-9413db2b9054 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:16.980149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1a55-cf52-4eb9-b71b-9413db2b9054 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:16.980415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1a55-cf52-4eb9-b71b-9413db2b9054 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:16.995099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda1a55-cf52-4eb9-b71b-9413db2b9054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-bfda1a55-cf52-4eb9-b71b-9413db2b9054 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:16.999953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:17.000822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:17.001990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 52/100] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:17.008769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.066063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:17.128928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64548b29-6308-4db2-8bd9-dc1f7f551db4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:17.174479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:17.175107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.175438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.175707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.176395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:17.176700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:17.178108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee563706-5e5a-4ec4-b363-9a7713483426 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:17.186925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2b776-bdf2-4257-9c03-7ee4c89304f0 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.187186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2b776-bdf2-4257-9c03-7ee4c89304f0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.187413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2b776-bdf2-4257-9c03-7ee4c89304f0 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.188765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 53/101] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:27:17.190215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 49/102] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:17.195853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2b776-bdf2-4257-9c03-7ee4c89304f0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.213449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887461f5-f14e-4f46-9e24-3765e43052c8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.213744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887461f5-f14e-4f46-9e24-3765e43052c8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.214012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887461f5-f14e-4f46-9e24-3765e43052c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.251457 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-887461f5-f14e-4f46-9e24-3765e43052c8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:17.252670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-887461f5-f14e-4f46-9e24-3765e43052c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:17.368018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2b776-bdf2-4257-9c03-7ee4c89304f0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:17.368802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 50/103] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:17.416219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350dacd-51bc-4028-ab07-e91624b7be15 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.416534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350dacd-51bc-4028-ab07-e91624b7be15 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.416788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350dacd-51bc-4028-ab07-e91624b7be15 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.418470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 51/104] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:17.423699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.423988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.424268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.434917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:17.435577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.435853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.436127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.437716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:17.439167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:17.441539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 54/105] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:17.442401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.462328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:17.463556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:17.514100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:17.526509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:17.571248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 52/106] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:17.572227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.611942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1da03f-2c85-4051-9eb9-06e06c784647 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:17.654677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4acd5d-1844-44f8-bf32-77d75895d7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:17.655325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4acd5d-1844-44f8-bf32-77d75895d7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.655603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4acd5d-1844-44f8-bf32-77d75895d7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.655886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4acd5d-1844-44f8-bf32-77d75895d7f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.666541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4acd5d-1844-44f8-bf32-77d75895d7f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ae4acd5d-1844-44f8-bf32-77d75895d7f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:17.671822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:17.673122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:17.674313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 55/107] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:17.676640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b391578-739d-465c-b76e-8f8c73d5ef6f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.676640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b391578-739d-465c-b76e-8f8c73d5ef6f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.676640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b391578-739d-465c-b76e-8f8c73d5ef6f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.677939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 56/108] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:17.679353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b391578-739d-465c-b76e-8f8c73d5ef6f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.684572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fa2fb-b230-4d31-a1ef-1ea3ef580171 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:17.732068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:17.732631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.732911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.733186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.734371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:17.736601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:17.739406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 53/109] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:17.739968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.782627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:17.787225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b391578-739d-465c-b76e-8f8c73d5ef6f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:17.802094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:17.829414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893caab0-ad07-4168-9174-d9bef0fd0ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:17.830448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893caab0-ad07-4168-9174-d9bef0fd0ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.831128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893caab0-ad07-4168-9174-d9bef0fd0ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.831620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893caab0-ad07-4168-9174-d9bef0fd0ff5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.844017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893caab0-ad07-4168-9174-d9bef0fd0ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-893caab0-ad07-4168-9174-d9bef0fd0ff5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:17.850749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:17.852495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:17.854215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 57/110] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:17.856929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.856929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.856929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.874077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dad9a-b6bf-4850-95e9-bc663567c305 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:17.881444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:17.883162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:17.916319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:17.917045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:17.917553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:17.917935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:17.919236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:17.919669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:17.921226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:17.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 54/111] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:17.932936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:17.965132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 58/112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:17.966122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.034693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.037794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cfb3f-69b8-4ad9-a3db-8c62fc0bc24b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.051888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.062038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622e889-32e1-4e9b-a51f-65d7cc25cf1e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.073221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.073899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.074210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.074494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562/users/186bbe5bc1074e8890ca95419df7673e/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.076546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] RBAC: Authorizing `identity:create_grant(project_id=32b9c55705c240fa9811f2211ec09562, user_id=186bbe5bc1074e8890ca95419df7673e, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.077951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.092385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 55/113] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:17 2022] PUT /identity/v3/projects/32b9c55705c240fa9811f2211ec09562/users/186bbe5bc1074e8890ca95419df7673e/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:18.106387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.106527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.106527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.106527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.106527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.106883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:18.106883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.106883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13498bfc-6a8a-41ab-9ea0-a2baa92dbfc1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.114856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 59/114] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:18.116285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63196a89-4613-4c27-b251-d2c1461e2c0b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.116577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63196a89-4613-4c27-b251-d2c1461e2c0b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.116854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63196a89-4613-4c27-b251-d2c1461e2c0b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.118343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 60/115] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:18.119447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b13023-105a-44ee-beb7-e0f54172d0f3 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.119730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b13023-105a-44ee-beb7-e0f54172d0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.120005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b13023-105a-44ee-beb7-e0f54172d0f3 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.121205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 61/116] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:18.122356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b13023-105a-44ee-beb7-e0f54172d0f3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.214139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55f5c0-5134-4d52-8e7d-e119b778154e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.232910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b13023-105a-44ee-beb7-e0f54172d0f3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.256122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.256327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.256491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.256771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.257929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.259656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.261523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 56/117] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:18.262270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.274758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83c3ce-4245-484c-aa74-e3b220ecf75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.275417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c3ce-4245-484c-aa74-e3b220ecf75a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.275703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c3ce-4245-484c-aa74-e3b220ecf75a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.275984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c3ce-4245-484c-aa74-e3b220ecf75a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.277119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83c3ce-4245-484c-aa74-e3b220ecf75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.278608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83c3ce-4245-484c-aa74-e3b220ecf75a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.280207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 62/118] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:18.281462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9e2df-d160-4f2f-a981-d0ac579f118d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.281754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9e2df-d160-4f2f-a981-d0ac579f118d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.282032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9e2df-d160-4f2f-a981-d0ac579f118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.293713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.300224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-61e9e2df-d160-4f2f-a981-d0ac579f118d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:18.301461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-61e9e2df-d160-4f2f-a981-d0ac579f118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:18.317366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.373241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce721e6b-101e-46d0-8f45-0b9c579ac0c4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.374982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 63/119] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:18.376120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.376420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.376698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.397819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:18.399087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:18.418131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.419091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.419193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.419455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.420593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.422275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.424112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 57/120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:18.429327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.467794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 64/121] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:18.468948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.549830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.552828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.559435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddcf5d-33e0-4286-abf0-1ff93be5f7c2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.573267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c5f7-b80a-4e28-ae5c-9d71020cd82b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.593358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.594001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.594328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.594605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.595717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.597523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.599198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 58/122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:18.602402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.615446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.616191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.616483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.616727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.617471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.619143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.648961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 65/123] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:27:18 2022] POST /identity/v3/projects => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:18.650281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.708594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841dea5-da61-4c10-8890-fba69df65126 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.754520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.756622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.757117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.757649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.774127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:18.782947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:18.783908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:18.784687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:18.787063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 59/124] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:18.788191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.797720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d5d4b-4f49-4fc0-8ecf-746b44f3d1ff None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.821008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.838962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:18.850131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.850131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.850131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.850131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.851559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:18.853588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:18.856955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 66/125] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:18.857101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:18.928203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d42ab3-fe7a-4a93-bf7c-d3850ed4334e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.977317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:18.979124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:18.979410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:18.979685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:18.981220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9974a-6ba6-4e44-bb54-938ffe40f332 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:18.995258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:19.002735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:19.003614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:19.004444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:19.006719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 60/126] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:19.008254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.019081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.019888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.020568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.020568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562/users/186bbe5bc1074e8890ca95419df7673e/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.023105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b9c55705c240fa9811f2211ec09562, user_id=186bbe5bc1074e8890ca95419df7673e, role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.024715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.045136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 67/127] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:18 2022] PUT /identity/v3/projects/32b9c55705c240fa9811f2211ec09562/users/186bbe5bc1074e8890ca95419df7673e/roles/14f7282e89144655a81a327a65c198a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:19.050362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.165108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:19.170695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:19.206489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383b1c3-1941-492d-97e1-ed62e6881c55 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.247891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09982da3-b390-46c2-b64c-182b1331a6b4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.276033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.276726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.277007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.277280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.277967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:19.278372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.279791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.296663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 61/128] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:19.297969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.312497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.313228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.313505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.314005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562/users/b1cb94a6532045fc84cecdb848b352fb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.317095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b9c55705c240fa9811f2211ec09562, user_id=b1cb94a6532045fc84cecdb848b352fb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.318658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.339827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 68/129] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:19 2022] PUT /identity/v3/projects/32b9c55705c240fa9811f2211ec09562/users/b1cb94a6532045fc84cecdb848b352fb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:19.340465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.468133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef220-ee3b-45b1-91f8-08903a8209bd None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.517707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa2853-76c6-454e-94f7-0e70d9b2a196 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.547890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.548878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.549351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.549838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.550899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:19.551419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.553044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.564327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 62/130] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:19.570601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.573395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:19.593389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:19.642911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.643586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.643871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.644148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.650440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.650440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.650440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 69/131] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:19 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:19.659120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.723721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d602f-1014-4891-b118-0d711d08f711 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.762611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.763273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.763605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.763888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] PATH_INFO: `/v3/users/2a1ecc551a2344989e9d6331c12c504d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.780092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] RBAC: Authorizing `identity:get_user(user_id=2a1ecc551a2344989e9d6331c12c504d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.781590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.783841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 63/132] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:19 2022] GET /identity/v3/users/2a1ecc551a2344989e9d6331c12c504d => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:19.789628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:19.865966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:19.909218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409622-d253-43f4-9062-9cc84f5dcc8b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.921744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5517f-7fd2-4763-acc0-ebe2a328db9e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:19.950644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.951338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.951631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.951918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.967949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:19.969068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:19.969542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:19.970009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] PATH_INFO: `/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:19.971552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] RBAC: Authorizing `identity:get_project(project_id=c2ac9703a4694dffbf4d792b7f0cf32b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:19.973830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:19.975971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 70/133] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:19 2022] GET /identity/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b => generated 312 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:19.987224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:20.043910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:20.045926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:20.115067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:20.122250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:20.168468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 64/134] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:20.176202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:20.213878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db314c33-6d14-451d-917b-9cbc8cb3c3cf None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:20.283822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742198f5-e92b-45c4-8f25-e1f33e76ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:20.284817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742198f5-e92b-45c4-8f25-e1f33e76ff24 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:20.285235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742198f5-e92b-45c4-8f25-e1f33e76ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:20.285644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742198f5-e92b-45c4-8f25-e1f33e76ff24 None admin] PATH_INFO: `/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b/users/186bbe5bc1074e8890ca95419df7673e/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:20.287738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742198f5-e92b-45c4-8f25-e1f33e76ff24 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2ac9703a4694dffbf4d792b7f0cf32b, user_id=186bbe5bc1074e8890ca95419df7673e, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:20.294543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742198f5-e92b-45c4-8f25-e1f33e76ff24 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:20.319106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 71/135] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:19 2022] PUT /identity/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b/users/186bbe5bc1074e8890ca95419df7673e/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:20.344844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f9c5b-5818-421d-bba5-e81c7cd91e9b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:20.345353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f9c5b-5818-421d-bba5-e81c7cd91e9b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:20.345896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f9c5b-5818-421d-bba5-e81c7cd91e9b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:20.347965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 72/136] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:20.354755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:20.355200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:20.355611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:20.357032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8784beb-5a70-42c4-8474-7fd09835f537 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:20.387252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:20.388485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:20.419147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:20.419830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:20.420113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:20.420389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:20.422845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:20.424322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:20.430725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 65/137] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:20 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:20.444393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:20.506521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:20.541526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 73/138] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:20.608682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:20.624337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86236c62-a615-42d8-8678-155cda6010f5 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:20.684506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:20.685140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:20.685427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:20.685707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:20.686482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] RBAC: Adding query filter params (role.id=050b752dca764106b2e5b9e1b2ff57b8, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=2a1ecc551a2344989e9d6331c12c504d) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:20.686801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:20.688875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:20.694908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 66/139] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:20 2022] GET /identity/v3/role_assignments?user.id=2a1ecc551a2344989e9d6331c12c504d&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=050b752dca764106b2e5b9e1b2ff57b8 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:20.697353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:20.780743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05779197-6ae5-4a01-8f97-5f3d12f7131f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:20.833126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:20.833835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:20.834119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:20.834431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:20.850521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:20.855371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:20.856236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:20.857028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:20.858114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 74/140] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:20.862949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.024571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:21.029349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656a80-5059-43b9-ab42-c057c74d828c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:21.076152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:21.077116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.077587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.078047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.078948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:21.079464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:21.081026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7c65a-745d-401f-ac20-2dbccdf3b442 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:21.093128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 67/141] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:21.099057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.192059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e239b1-ee0b-40bc-8cb3-de507b079ce9 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.192415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e239b1-ee0b-40bc-8cb3-de507b079ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.192710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e239b1-ee0b-40bc-8cb3-de507b079ce9 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.194268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 68/142] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:21.200667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474df20-3069-49be-954a-8ca0829db107 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.201194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474df20-3069-49be-954a-8ca0829db107 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.202339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474df20-3069-49be-954a-8ca0829db107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.232755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-3474df20-3069-49be-954a-8ca0829db107 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:21.233986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3474df20-3069-49be-954a-8ca0829db107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:21.244840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff86b7-26b2-4197-bb09-64e088f25b3f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:21.258613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:21.296294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:21.297330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.297811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.298331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.302846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:21.304564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:21.306524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 75/143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:21 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:21.312536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.381200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 69/144] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:21.450475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b551b7-07f0-4056-bb9a-607e5eb53698 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:21.478684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474df20-3069-49be-954a-8ca0829db107 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.491490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:21.492428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.492954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.493559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.537775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:21.564129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:21.565953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:21.659528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:21.666918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 76/145] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 02:27:21.676715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474df20-3069-49be-954a-8ca0829db107 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:21.679222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.737187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:21.737832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.738107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.738401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.740512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:21.741958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:21.743835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 70/146] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:21.750962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.805828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:21.859967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e464-eeb8-4afe-a4f8-bb095da692bf None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:21.889758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cbeb3-296d-4c5b-ba18-23863f16a552 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:21.924208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435ad2b-999d-4e71-856c-c185a074fc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:21.925176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ad2b-999d-4e71-856c-c185a074fc33 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.925651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ad2b-999d-4e71-856c-c185a074fc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.926134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ad2b-999d-4e71-856c-c185a074fc33 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.927567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ad2b-999d-4e71-856c-c185a074fc33 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:21.929232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ad2b-999d-4e71-856c-c185a074fc33 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:21.931615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 77/147] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:21 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:21.932944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388f31-e226-401a-9e75-51e8be2b7887 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.938424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388f31-e226-401a-9e75-51e8be2b7887 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.938927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388f31-e226-401a-9e75-51e8be2b7887 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.940443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 78/148] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:21.941465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388f31-e226-401a-9e75-51e8be2b7887 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:21.950958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c981fd-c99c-4523-9262-4a0703d98b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:21.951608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c981fd-c99c-4523-9262-4a0703d98b7f None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:21.951889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c981fd-c99c-4523-9262-4a0703d98b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:21.952162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c981fd-c99c-4523-9262-4a0703d98b7f None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:21.952843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c981fd-c99c-4523-9262-4a0703d98b7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:21.954330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c981fd-c99c-4523-9262-4a0703d98b7f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:22.006620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 71/149] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:21 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:27:22.014453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.014453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.014453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.059365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:22.060717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:22.072887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:22.088261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388f31-e226-401a-9e75-51e8be2b7887 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:22.153920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:22.155795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.157157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.157627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.158825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=186bbe5bc1074e8890ca95419df7673e) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:22.160144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:22.161962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:22.176226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 79/150] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:21 2022] GET /identity/v3/role_assignments?user.id=186bbe5bc1074e8890ca95419df7673e&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:22.201984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 72/151] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:22.244604 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:22.262181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:22.383111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41910db-79d0-49e0-8c07-701737eef5af None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:22.442636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:22.443712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.444194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.444716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.457929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:22.463896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:22.464836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:22.465631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:22.468124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 80/152] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:22.479365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:22.609460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ada9d-2a2b-4640-baa5-7b8b59198595 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:22.648868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:22.649601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.649880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.650165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.650881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:22.651186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:22.652574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b576cf-d5d9-4de3-972e-1a5c8babedee None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:22.663333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 73/153] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:22.668823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:22.738499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:22.761020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4dda-ecc4-4623-ba5b-0a818b3d6723 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.761368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4dda-ecc4-4623-ba5b-0a818b3d6723 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.761647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4dda-ecc4-4623-ba5b-0a818b3d6723 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.763715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 74/154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:22.773720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.774444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.774824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.781964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c9b41-d815-4643-9c26-dbbdf385d0f6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:22.794613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:22.800541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:22.801774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:22.854029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:22.854723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:22.855012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:22.855277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:22.856439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:22.857972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:22.859785 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 81/155] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:22 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:22.864953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:22.894769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 75/156] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:22.917776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:22.985570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6173e7-b10d-4adc-a48a-981fb088a783 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.031117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.031896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.032176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.032870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.036638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:23.043385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b4395-d642-49dd-bd86-f465ac80cda8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.093839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.094521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.094797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.095063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.101947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:23.116821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:23.122335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:23.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:23.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 76/157] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:23.134229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:23.136304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:23.137807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:23.244702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa12c7-7f25-4ef3-8abf-4a6f94101538 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.257368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 82/158] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:23.262988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:23.282898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.283541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.283827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.284101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.284851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:23.285214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:23.286893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:23.300781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 77/159] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:23.302248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:23.308265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:23.367647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:23.431481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4f6e3-7673-4549-ac6b-20c347fae32a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.436360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38cd57-7d1b-4ff0-a529-3dc4d29ab4a2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.480288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46720be5-b23b-49f3-ac4b-649015c4f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.481295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720be5-b23b-49f3-ac4b-649015c4f8eb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.481770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720be5-b23b-49f3-ac4b-649015c4f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.482264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720be5-b23b-49f3-ac4b-649015c4f8eb None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.483740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46720be5-b23b-49f3-ac4b-649015c4f8eb None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:23.484738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.485433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.485719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.485992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] PATH_INFO: `/v3/users/b1cb94a6532045fc84cecdb848b352fb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.486246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46720be5-b23b-49f3-ac4b-649015c4f8eb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:23.487220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] RBAC: Authorizing `identity:get_user(user_id=b1cb94a6532045fc84cecdb848b352fb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:23.488312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 83/160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:23 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:23.488737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:23.489903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254420d-f67a-4df1-a773-cda0562d2bc9 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.490409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254420d-f67a-4df1-a773-cda0562d2bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.490510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 78/161] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:23 2022] GET /identity/v3/users/b1cb94a6532045fc84cecdb848b352fb => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:23.490984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254420d-f67a-4df1-a773-cda0562d2bc9 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.492402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 84/162] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:23.494908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:23.498090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.498631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.499118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.520119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:23.521685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:23.604167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 85/163] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:23.605147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:23.634233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92689e-3be6-487d-898d-14d5d0be2da4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.643185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:23.672250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:23.683722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.685204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.685896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.686356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.687495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] RBAC: Adding query filter params (role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=186bbe5bc1074e8890ca95419df7673e) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:23.688858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:23.690594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24113d1e-647b-4045-af63-c432c0b68f9c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:23.700671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 79/164] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:23 2022] GET /identity/v3/role_assignments?user.id=186bbe5bc1074e8890ca95419df7673e&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=14f7282e89144655a81a327a65c198a5 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:23.701724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eb9bb-d4c9-47a9-8f78-fdb6481c53fe None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.702010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eb9bb-d4c9-47a9-8f78-fdb6481c53fe None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.705171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eb9bb-d4c9-47a9-8f78-fdb6481c53fe None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.706796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 80/165] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:23.708566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eb9bb-d4c9-47a9-8f78-fdb6481c53fe None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:23.749829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f9ce-d35b-4ec5-b11f-6052b20b7c8e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.807513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88fcb-dcdd-4a83-83e7-2b183b668924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:23.808528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88fcb-dcdd-4a83-83e7-2b183b668924 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:23.809007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88fcb-dcdd-4a83-83e7-2b183b668924 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:23.809477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88fcb-dcdd-4a83-83e7-2b183b668924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:23.888021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88fcb-dcdd-4a83-83e7-2b183b668924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:23.889879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88fcb-dcdd-4a83-83e7-2b183b668924 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:23.923912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eb9bb-d4c9-47a9-8f78-fdb6481c53fe None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:23.941464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:24.020295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:24.020954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.021236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.021509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.028433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 86/166] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:24.029502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.029790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.030064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.048324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:24.049521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:24.057534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:24.062824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:24.063745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 81/167] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:24.065391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.117592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 87/168] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:24.118634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.183544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:24.199312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:24.240649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6afda-f7a0-499c-9d04-b20ef42861e0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:24.252495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f7358-f473-46d7-85b5-33586f8df32e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:24.278034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:24.278769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.279008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.279330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.280006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:24.280302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:24.281686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:24.291587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 88/169] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:24.292414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.304268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:24.304921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.305198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.305533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.309233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:24.311015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:24.312987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 82/170] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:24.314101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.418618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b7f01-93b1-4979-ab72-457efb0f1cf7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:24.454994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:24.456500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:24.457682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.458209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.458635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.470765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:24.477084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:24.478168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:24.479050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:24.480964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 89/171] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:24.481864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.692120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36eee9d-3a73-49c7-8aa3-7ef214498d3a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:24.694930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f3233-8086-4fc1-a163-82c05f905177 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:24.758098 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:24.773520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:24.774617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.775101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.775583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.776532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=b1cb94a6532045fc84cecdb848b352fb) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:24.777037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:24.779065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:24.789216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 90/172] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/role_assignments?user.id=b1cb94a6532045fc84cecdb848b352fb&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:24.792009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.823780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:24.824478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:24.824761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:24.825033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] PATH_INFO: `/v3/users/186bbe5bc1074e8890ca95419df7673e` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:24.826220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] RBAC: Authorizing `identity:get_user(user_id=186bbe5bc1074e8890ca95419df7673e)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:24.827786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:24.829401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 83/173] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/users/186bbe5bc1074e8890ca95419df7673e => generated 275 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:24.837279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:24.876252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:25.045978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcda613-b25f-4c5d-992d-c5c2f958cc94 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.139842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.142048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.142356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.142622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.143313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:25.143616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.145018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.160331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 91/174] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:25.167178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:25.189926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a667f0c-df55-458f-aa97-e8a4c47daf97 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.262001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.267716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.267716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.267716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.300338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42a02f-bee9-4659-91a9-d431dc04ce35 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.312617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:25.339377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.340504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.341122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.341609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] PATH_INFO: `/v3/users/2a1ecc551a2344989e9d6331c12c504d` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.343030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] RBAC: Authorizing `identity:get_user(user_id=2a1ecc551a2344989e9d6331c12c504d)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.344751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.346625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 92/175] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:25 2022] GET /identity/v3/users/2a1ecc551a2344989e9d6331c12c504d => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:25.348720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:25.352956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:25.372778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.372778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.477675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 84/176] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:25.484917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:25.544108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64e89-415e-462d-8ab9-356a1f39bc61 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.589369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15171ba0-768d-48f7-80e0-f46ee3b803f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.590309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15171ba0-768d-48f7-80e0-f46ee3b803f9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.590603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15171ba0-768d-48f7-80e0-f46ee3b803f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.590883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15171ba0-768d-48f7-80e0-f46ee3b803f9 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.592638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:25.593317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15171ba0-768d-48f7-80e0-f46ee3b803f9 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.594852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15171ba0-768d-48f7-80e0-f46ee3b803f9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.597863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 93/177] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:25 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:25.599876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd896-189f-419e-bae7-e7f11e6eec66 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.600168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd896-189f-419e-bae7-e7f11e6eec66 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.600441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd896-189f-419e-bae7-e7f11e6eec66 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.601676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 94/178] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:25.603265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd896-189f-419e-bae7-e7f11e6eec66 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:25.617964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccbb6d-1c87-45c2-a7e7-1fa10b574776 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.655587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce6721-843a-40e9-8f5c-ab26859a8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.656340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce6721-843a-40e9-8f5c-ab26859a8e23 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.656692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce6721-843a-40e9-8f5c-ab26859a8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.657030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce6721-843a-40e9-8f5c-ab26859a8e23 None admin] PATH_INFO: `/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.658410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce6721-843a-40e9-8f5c-ab26859a8e23 None admin] RBAC: Authorizing `identity:get_project(project_id=c2ac9703a4694dffbf4d792b7f0cf32b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.660031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce6721-843a-40e9-8f5c-ab26859a8e23 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.666972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 85/179] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:25 2022] GET /identity/v3/projects/c2ac9703a4694dffbf4d792b7f0cf32b => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:25.667937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:25.669565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.670072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.671294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.690948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:25.692403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:25.736776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd896-189f-419e-bae7-e7f11e6eec66 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.778182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 86/180] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:25.778182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:25.781808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.782809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.783311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.783797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562/users/2a1ecc551a2344989e9d6331c12c504d/roles/050b752dca764106b2e5b9e1b2ff57b8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.785948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] RBAC: Authorizing `identity:create_grant(project_id=32b9c55705c240fa9811f2211ec09562, user_id=2a1ecc551a2344989e9d6331c12c504d, role_id=050b752dca764106b2e5b9e1b2ff57b8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.791791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.817328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 95/181] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:25 2022] PUT /identity/v3/projects/32b9c55705c240fa9811f2211ec09562/users/2a1ecc551a2344989e9d6331c12c504d/roles/050b752dca764106b2e5b9e1b2ff57b8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:25.823431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:25.876749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:25.898962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33176efe-e983-45ed-9e9a-38ec72190198 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:25.941115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:25.941757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:25.942037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:25.942335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:25.943693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=c2ac9703a4694dffbf4d792b7f0cf32b, user.id=186bbe5bc1074e8890ca95419df7673e) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:25.944010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:25.945699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76542e4-0ea5-4d31-bd5d-89d155e51c3b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:25.954691 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 87/182] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:25 2022] GET /identity/v3/role_assignments?user.id=186bbe5bc1074e8890ca95419df7673e&scope.project.id=c2ac9703a4694dffbf4d792b7f0cf32b&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:26.028041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7efbf-e56d-4b2d-b97c-a5352b37f11c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.041741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.091278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.091943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.092224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.092492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.107773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:26.107965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213cb85-f105-489c-bbc6-7a42e08b97d9 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.108046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213cb85-f105-489c-bbc6-7a42e08b97d9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.108366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213cb85-f105-489c-bbc6-7a42e08b97d9 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.109892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 88/183] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:26.112630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:26.113527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:26.114431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 96/184] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:26.117500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4397-e907-4aee-90e7-7156e433664d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.117953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4397-e907-4aee-90e7-7156e433664d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.118350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4397-e907-4aee-90e7-7156e433664d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.119741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.146141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6d2f4397-e907-4aee-90e7-7156e433664d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:26.148182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6d2f4397-e907-4aee-90e7-7156e433664d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:26.233060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 89/185] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:26.234132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b5bc3-92af-4bcb-b8e8-06b1b4a832f6 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.234478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b5bc3-92af-4bcb-b8e8-06b1b4a832f6 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.234735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b5bc3-92af-4bcb-b8e8-06b1b4a832f6 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.236256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 90/186] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:26.241694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.241987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.242334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.262228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:26.263518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:26.272253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c9c4d-b032-47c5-8463-10100930f409 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.318393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.319211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.319501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.319795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.320483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:26.320790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.322298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.330869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 97/187] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:26.336071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.341073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 91/188] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:26.341825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.361301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.376877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.461740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f37091-3eee-48b0-9eb7-126e79569708 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.461965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091d28-11ae-4e90-bf91-a1529eddee77 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.495200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.495845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.496135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.496415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] PATH_INFO: `/v3/users/21f188818fe74c5fb8aef4d7989168b0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.498432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.499462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.499780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.500048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.509956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] RBAC: Authorizing `identity:get_user(user_id=21f188818fe74c5fb8aef4d7989168b0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.511318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:26.511449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.513306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 92/189] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/users/21f188818fe74c5fb8aef4d7989168b0 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:26.513987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:26.521226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:26.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:26.522948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 98/190] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:26.523621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.584427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.619041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.619707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a258f3b-4a74-44ab-b3c7-977d606d18a7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.634557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5b1e0-11dd-42eb-a89d-5ea42a776208 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.655026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.655667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.655946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.656218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.657330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.658818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.660577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 93/191] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:26.666251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.674185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.674822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.675102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.675371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.744983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.746483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.777553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a680f68-2e9a-4cf8-b6e6-fadeb17cc5ce None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.812537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.813159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.813429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.813696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.814395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:26.814701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.816073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.817432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.825900 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 94/192] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:26.826762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.832692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:26.846624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 99/193] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:26.848120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:26.937403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f408c-c893-4f56-b82e-6e6c1b403406 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.947692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109c337-e889-4840-a38e-c6d9175d082c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:26.972955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.973573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.973861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.974135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.974811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.976289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.982473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:26.983108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:26.983386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:26.983655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] PATH_INFO: `/v3/users/21f188818fe74c5fb8aef4d7989168b0` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:26.984774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] RBAC: Authorizing `identity:get_user(user_id=21f188818fe74c5fb8aef4d7989168b0)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:26.986302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:26.988704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 100/194] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/users/21f188818fe74c5fb8aef4d7989168b0 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:26.990265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:27.005565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 95/195] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:26 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:27.005565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:27.072074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:27.073057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:27.127685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd914a-2578-423a-a1d0-0e727313ed88 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:27.141420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b7cee-0264-4c59-b388-023f79683e0c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:27.165880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:27.166793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:27.167234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:27.167584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:27.168900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:27.170682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:27.172925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 101/196] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:26 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:27.181788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:27.182790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:27.183242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:27.183685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:27.187244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:27.272810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:27.274348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:27.319356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e0950-d01f-44e1-89af-5e2ba8f224da None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:27.341616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:27.352453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:27.369283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:27.369915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:27.370221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:27.370502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:27.371248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=21f188818fe74c5fb8aef4d7989168b0) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:27.371555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:27.373173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:27.380408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 102/197] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:27 2022] GET /identity/v3/role_assignments?user.id=21f188818fe74c5fb8aef4d7989168b0&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:27.420688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 96/198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:27.427015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:27.549191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29a12f-6f65-4a02-88fe-6a8798b7d5e5 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:27.583778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d90df1-13a5-4942-9daa-98ccb907baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:27.584413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d90df1-13a5-4942-9daa-98ccb907baf9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:27.584689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d90df1-13a5-4942-9daa-98ccb907baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:27.584957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d90df1-13a5-4942-9daa-98ccb907baf9 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:27.586099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d90df1-13a5-4942-9daa-98ccb907baf9 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:27.587644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d90df1-13a5-4942-9daa-98ccb907baf9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:27.589480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 103/199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:27 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:27.595278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:27.713933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84977271-cc19-48d3-b19b-1fe819d8082f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:27.763514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:27.764151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:27.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:27.764707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:27.765460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] RBAC: Adding query filter params (role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=21f188818fe74c5fb8aef4d7989168b0) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:27.765771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:27.767272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bb334-835f-45ba-a9ce-aefdb36148dd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:27.778434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 97/200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:27 2022] GET /identity/v3/role_assignments?user.id=21f188818fe74c5fb8aef4d7989168b0&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=14f7282e89144655a81a327a65c198a5 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:28.126205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a231c1-3681-4c55-bb5f-c0c7069ecd3b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:28.126615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a231c1-3681-4c55-bb5f-c0c7069ecd3b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:28.126791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a231c1-3681-4c55-bb5f-c0c7069ecd3b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:28.128172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 104/201] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:28.135816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:28.135816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:28.135816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:28.157865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:28.159456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:28.171239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:28.242624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 98/202] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:28.277421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a231c1-3681-4c55-bb5f-c0c7069ecd3b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:28.347859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:28.425607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a231c1-3681-4c55-bb5f-c0c7069ecd3b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:28.469948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:28.470617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:28.470896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:28.471164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:28.487033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:28.491827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:28.492706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:28.493708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 105/203] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:28.499181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:28.610166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5974f-28b8-4e51-b8ea-d35f17f06b14 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:28.648703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:28.649812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:28.650351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:28.650814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:28.651680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:28.652184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:28.653771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:28.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 99/204] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:28.668832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:28.784175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a0693-7873-4b54-90d4-2a7724c4a9f6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:28.806186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:28.832490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:28.833144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:28.833425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:28.833699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] PATH_INFO: `/v3/users/2a1ecc551a2344989e9d6331c12c504d` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:28.835126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] RBAC: Authorizing `identity:get_user(user_id=2a1ecc551a2344989e9d6331c12c504d)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:28.836750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:28.838572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 106/205] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:28 2022] GET /identity/v3/users/2a1ecc551a2344989e9d6331c12c504d => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:28.844099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:28.882330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:28.978985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9434-5f1b-415b-84ed-5a69de5b9462 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:29.017092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:29.017984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.018500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.018961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.098664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:29.100486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:29.118182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:29.225304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 100/206] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:29.235346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:29.385949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6620-e838-479c-9f1b-e4b321dadfec None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:29.444767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6dd01-018a-45e7-8f93-31d1110912fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:29.445423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6dd01-018a-45e7-8f93-31d1110912fd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.445705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6dd01-018a-45e7-8f93-31d1110912fd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.446025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6dd01-018a-45e7-8f93-31d1110912fd None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.447240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6dd01-018a-45e7-8f93-31d1110912fd None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:29.448748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6dd01-018a-45e7-8f93-31d1110912fd None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:29.450457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 107/207] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:29 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:29.456390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:29.560824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c820c8-6475-49ea-a2e6-fd45c378a3e0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:29.571192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:29.603005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38036812-e39e-4090-b55d-24339563a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:29.603622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38036812-e39e-4090-b55d-24339563a290 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.603899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38036812-e39e-4090-b55d-24339563a290 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.604165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38036812-e39e-4090-b55d-24339563a290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.604899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38036812-e39e-4090-b55d-24339563a290 None admin] RBAC: Adding query filter params (role.id=050b752dca764106b2e5b9e1b2ff57b8, scope.project.id=32b9c55705c240fa9811f2211ec09562, user.id=2a1ecc551a2344989e9d6331c12c504d) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:29.605201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38036812-e39e-4090-b55d-24339563a290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:29.606658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38036812-e39e-4090-b55d-24339563a290 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:29.612746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 101/208] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:29 2022] GET /identity/v3/role_assignments?user.id=2a1ecc551a2344989e9d6331c12c504d&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=050b752dca764106b2e5b9e1b2ff57b8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:29.719489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248103f8-d178-4e9f-8ddf-25ab347c32fc None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.720155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248103f8-d178-4e9f-8ddf-25ab347c32fc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.720626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248103f8-d178-4e9f-8ddf-25ab347c32fc None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.722681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 108/209] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:29.728824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.729454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.729923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.758709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:29.760298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:29.864630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 102/210] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:29.910857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88a43-4564-47ed-b8e7-85de596ac619 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.911487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88a43-4564-47ed-b8e7-85de596ac619 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.911958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88a43-4564-47ed-b8e7-85de596ac619 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.913598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 109/211] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:29.915028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:29.920246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308fb71-b3d9-4c05-a9e9-ccebcc7ba871 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:29.920784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308fb71-b3d9-4c05-a9e9-ccebcc7ba871 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:29.921259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308fb71-b3d9-4c05-a9e9-ccebcc7ba871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:29.955315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a308fb71-b3d9-4c05-a9e9-ccebcc7ba871 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:29.956912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a308fb71-b3d9-4c05-a9e9-ccebcc7ba871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:30.015060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.028698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aca9cb-9671-4a3d-94cf-1edb694c21db None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.033656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.046014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 110/212] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:30.047743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce95ec-b8ca-4deb-a274-e2391be6a17d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.048245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce95ec-b8ca-4deb-a274-e2391be6a17d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.048629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce95ec-b8ca-4deb-a274-e2391be6a17d None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.050126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 111/213] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:30.055588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.055878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.056159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.075734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.076689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:30.076972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.077438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.077887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:30.078103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.090961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:30.095908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:30.096822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:30.098000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 103/214] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:30.099100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.159562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 112/215] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:30.160195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.232880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b1033-9d82-4e86-9741-008d08586196 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.266800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.267444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.267723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.268069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.269090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.270585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:30.276758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 104/216] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:30.276892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827dbab-2528-4793-8c96-80b1dc3099ae None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.276994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.290059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.305779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.310950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.311566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.311843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.312109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.312771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:30.313072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.314479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:30.322443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 113/217] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:30.323387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.378765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823e9f7-57d7-427e-b902-e3849ddeac51 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.415674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654811-df3d-483d-bb96-8641a1a5584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.416324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654811-df3d-483d-bb96-8641a1a5584e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.416605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654811-df3d-483d-bb96-8641a1a5584e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.416898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654811-df3d-483d-bb96-8641a1a5584e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.425355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a559c-2ce0-48c0-91d8-41104655129d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.427349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654811-df3d-483d-bb96-8641a1a5584e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-83654811-df3d-483d-bb96-8641a1a5584e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:30.432177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:30.433370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:30.434227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 105/218] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:30.434921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1e12b-f23b-4a50-b53d-3da456efa697 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.434921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1e12b-f23b-4a50-b53d-3da456efa697 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.435049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1e12b-f23b-4a50-b53d-3da456efa697 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.436382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 106/219] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:30.436919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1e12b-f23b-4a50-b53d-3da456efa697 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.463766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.464379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.464650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.464916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.465553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.467038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:30.488473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 114/220] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:30 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:30.495785 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.528884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.540379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.554637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1e12b-f23b-4a50-b53d-3da456efa697 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.592904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fb728-fda1-4cd9-9f43-5f0ea35dbc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.593626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fb728-fda1-4cd9-9f43-5f0ea35dbc99 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.593813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fb728-fda1-4cd9-9f43-5f0ea35dbc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.594080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fb728-fda1-4cd9-9f43-5f0ea35dbc99 None admin] PATH_INFO: `/v3/users/a1f5a8ba16f1409ca73a2ea70c44ea9d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.610063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fb728-fda1-4cd9-9f43-5f0ea35dbc99 None admin] RBAC: Authorizing `identity:get_user(user_id=a1f5a8ba16f1409ca73a2ea70c44ea9d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.611671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fb728-fda1-4cd9-9f43-5f0ea35dbc99 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:30.613670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 107/221] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/users/a1f5a8ba16f1409ca73a2ea70c44ea9d => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:30.614922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.615212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.615490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.626109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec009d-fc70-4684-b379-1131a74d4312 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.636662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:30.638313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:30.666011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.666570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.666848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.667114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.667806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:30.668128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.669566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:30.678141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 115/222] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:30.678961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.731587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 108/223] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:30.732751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.786511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.796981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4f868-df67-4dc8-9230-c1111043442d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.822681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:30.837704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.838957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.838957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.838957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.852891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d1976-7269-4722-9a0a-15f2e1041448 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:30.900138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:30.901192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:30.901609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:30.901960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] PATH_INFO: `/v3/users/21f188818fe74c5fb8aef4d7989168b0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:30.903634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] RBAC: Authorizing `identity:get_user(user_id=21f188818fe74c5fb8aef4d7989168b0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.905268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:30.907556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 109/224] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/users/21f188818fe74c5fb8aef4d7989168b0 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:30.909431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:30.929437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:30.931329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.026699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75d7ef-7596-43c0-ba29-abb5a618abfc None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.027993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:31.037261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 116/225] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:31.038415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.064632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.065088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.065364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.065632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.076932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:31.081950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:31.082908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:31.083754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 110/226] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:31.084588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.098510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:31.173734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2b79-b2e2-4a37-a1fa-fafe0891a2f2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.201349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012edda-fe98-4c24-b62a-353e45a6c7be None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.219348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.220006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.220317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.220995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.222517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.223993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.230291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 117/227] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:31 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:31.231404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.246916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.247981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.248272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.248553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.249992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.251779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.254602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 111/228] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:31 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:31.255698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.298557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:31.346703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f4c53-35f7-4c6c-9ae5-0b33429c7235 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.361606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:31.385609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.386722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.387248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.387810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.388686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:31.389201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.391127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.394180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2095ec-cd19-473a-a5c2-0a60f6ab8048 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.400595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 118/229] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:31.401676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.442580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.442936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.443138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.443412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/21f188818fe74c5fb8aef4d7989168b0/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.448964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f446bfe68eb142619642d064c9ee2a41, user_id=21f188818fe74c5fb8aef4d7989168b0, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.450259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.468495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 112/230] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:31 2022] PUT /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/21f188818fe74c5fb8aef4d7989168b0/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:31.476217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.613098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b594a70-883b-48ac-bd38-5b24e08d8742 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.660108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:31.677190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f173cdf-2804-43f5-a958-357774468bd4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.684235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.684871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.685158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.685435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.686226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=a1f5a8ba16f1409ca73a2ea70c44ea9d) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:31.686545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.688009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.696324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 119/231] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:31 2022] GET /identity/v3/role_assignments?user.id=a1f5a8ba16f1409ca73a2ea70c44ea9d&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:31.728074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.729006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.729432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.729904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] PATH_INFO: `/v3/users/21f188818fe74c5fb8aef4d7989168b0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.731448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] RBAC: Authorizing `identity:get_user(user_id=21f188818fe74c5fb8aef4d7989168b0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.733221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.735735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 113/232] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:31 2022] GET /identity/v3/users/21f188818fe74c5fb8aef4d7989168b0 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:31.742310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.885418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02fba6-f8b6-4d6c-bccc-dcb8a48684a4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:31.919652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9662f8f-36ed-4ae2-a27f-b6cd61228867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:31.920336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9662f8f-36ed-4ae2-a27f-b6cd61228867 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:31.920616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9662f8f-36ed-4ae2-a27f-b6cd61228867 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:31.920882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9662f8f-36ed-4ae2-a27f-b6cd61228867 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:31.922006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9662f8f-36ed-4ae2-a27f-b6cd61228867 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:31.923561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9662f8f-36ed-4ae2-a27f-b6cd61228867 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:31.925382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 120/233] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:31 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:31.931159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:31.936846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:32.060926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9ecc6-2f51-4b94-90d3-23e931e8ba5f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:32.106643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff64973-e4e7-4bcc-83db-aa0afddef1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:32.107377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff64973-e4e7-4bcc-83db-aa0afddef1b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:32.107659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff64973-e4e7-4bcc-83db-aa0afddef1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:32.107933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff64973-e4e7-4bcc-83db-aa0afddef1b3 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/21f188818fe74c5fb8aef4d7989168b0/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:32.110202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff64973-e4e7-4bcc-83db-aa0afddef1b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f446bfe68eb142619642d064c9ee2a41, user_id=21f188818fe74c5fb8aef4d7989168b0, role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:32.111647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff64973-e4e7-4bcc-83db-aa0afddef1b3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:32.129696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 114/234] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:31 2022] PUT /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/21f188818fe74c5fb8aef4d7989168b0/roles/14f7282e89144655a81a327a65c198a5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:32.859470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dccb4e-3cde-431d-ab48-f6e9d9db209f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:32.859994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dccb4e-3cde-431d-ab48-f6e9d9db209f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:32.859994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dccb4e-3cde-431d-ab48-f6e9d9db209f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:32.861608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 121/235] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:32.867162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee72eb-0d6d-40a7-a26f-b693b0b372bc None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:32.867574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee72eb-0d6d-40a7-a26f-b693b0b372bc None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:32.867758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee72eb-0d6d-40a7-a26f-b693b0b372bc None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:32.868290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:32.868579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:32.868858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:32.869125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 115/236] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:32.884373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:32.884772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:32.885046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:32.890260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:32.891655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:32.923421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:32.924640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:32.955435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:32.991888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 122/237] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:33.014392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:33.023269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:33.034991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 116/238] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:33.070687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:33.181639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436a5ba-e335-4762-8728-54b34ee1ce44 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:33.255782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:33.258115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.258741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.259212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.279850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cafe1-af17-42ee-ac6c-064f79b63ea6 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:33.286793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:33.294599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:33.295711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:33.296719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:33.300185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 123/239] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:33.306117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:33.340828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:33.341708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.342131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.342571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.344574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:33.346226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:33.350635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 117/240] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:33.356556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:33.422505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:33.435576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:33.481658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116837f-f0b5-46a3-a6c0-75ee8e2824da None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:33.531929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:33.532907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.533423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.533897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.534790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:33.536437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:33.566360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e59cd6-877f-4fba-9c74-dca542fa5132 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:33.582473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 118/241] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:33 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:33.646984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:33.647627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.647903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.648167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.648866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:33.649296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:33.650758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:33.667113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 124/242] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:33.672857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:33.793897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aed6de-4fa0-4625-931f-d7fb719e7ec0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:33.815603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:33.835211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa7b47-6a49-4756-bd51-a1fecc0b6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:33.835854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa7b47-6a49-4756-bd51-a1fecc0b6f14 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.836131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa7b47-6a49-4756-bd51-a1fecc0b6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.836400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa7b47-6a49-4756-bd51-a1fecc0b6f14 None admin] PATH_INFO: `/v3/users/7d601c5dd5ed4c47ae280c3e4450728b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.852184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa7b47-6a49-4756-bd51-a1fecc0b6f14 None admin] RBAC: Authorizing `identity:get_user(user_id=7d601c5dd5ed4c47ae280c3e4450728b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:33.853703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa7b47-6a49-4756-bd51-a1fecc0b6f14 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:33.855744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 119/243] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:33 2022] GET /identity/v3/users/7d601c5dd5ed4c47ae280c3e4450728b => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:33.861278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:33.929612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04bcaa-f862-431a-a06c-9a53e1e6355d None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.930140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04bcaa-f862-431a-a06c-9a53e1e6355d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.930140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04bcaa-f862-431a-a06c-9a53e1e6355d None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.931954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 120/244] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:33.937098 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:33.937551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:33.937781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:33.958993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:33.960190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:33.983383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6fe2c-c492-4315-962a-0eb8190ae579 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:33.993082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.028140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad5782-8a69-4388-a86e-3212026078d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.028848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad5782-8a69-4388-a86e-3212026078d6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.029133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad5782-8a69-4388-a86e-3212026078d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.029416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad5782-8a69-4388-a86e-3212026078d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.049315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 121/245] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:34.074033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.106341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad5782-8a69-4388-a86e-3212026078d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:34.108327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad5782-8a69-4388-a86e-3212026078d6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:34.219242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47da56-d40e-44d8-9ef4-36b868c44226 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:34.221646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.238196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 125/246] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:34.239773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77f9a-7608-4f94-ab82-872986319389 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.240188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77f9a-7608-4f94-ab82-872986319389 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.240618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77f9a-7608-4f94-ab82-872986319389 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.242512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 126/247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:34.243408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77f9a-7608-4f94-ab82-872986319389 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.257303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.267084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bfec2-4cc2-47b9-a983-d05b185a558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.267737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bfec2-4cc2-47b9-a983-d05b185a558d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.268016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bfec2-4cc2-47b9-a983-d05b185a558d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.268287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bfec2-4cc2-47b9-a983-d05b185a558d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.279957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bfec2-4cc2-47b9-a983-d05b185a558d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-932bfec2-4cc2-47b9-a983-d05b185a558d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:34.284756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:34.285686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:34.286642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 122/248] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:34.287927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.287927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.288082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.307513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:34.308646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:34.395663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77f9a-7608-4f94-ab82-872986319389 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:34.423934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 123/249] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:34.424840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.471729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.473024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.473304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.473576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.476011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:34.477514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:34.482400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 127/250] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:34.483841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.581127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.588051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37f17-e83d-4a06-ac26-d0f9e25c8bda None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:34.622904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.623583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.623866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.624139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.624813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:34.625162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:34.626642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:34.630886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.636011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 124/251] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:34.642326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.674911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101935-55cd-431c-9e3a-f6b7adcfbd12 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:34.710080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.710874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.711151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.711529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.722088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:34.728189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:34.729078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:34.729902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 128/252] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:34.730607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.745870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d41f0-e722-4a98-b455-0fd5f7295831 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:34.787617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.790274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.790567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.790870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.791653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] RBAC: Adding query filter params (role.id=050b752dca764106b2e5b9e1b2ff57b8, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=7d601c5dd5ed4c47ae280c3e4450728b) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:34.791965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:34.793474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41f5d-a547-4827-8bd0-264320cc95d6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:34.799772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 125/253] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/role_assignments?user.id=7d601c5dd5ed4c47ae280c3e4450728b&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=050b752dca764106b2e5b9e1b2ff57b8 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:34.800823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022991af-4efb-40aa-9d5d-f829d95f110e None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.801109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022991af-4efb-40aa-9d5d-f829d95f110e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.801385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022991af-4efb-40aa-9d5d-f829d95f110e None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.810931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 126/254] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:34.811850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022991af-4efb-40aa-9d5d-f829d95f110e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:34.840733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cbeb5-148e-48b0-888d-71225e104fb2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:34.866169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.886539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084309c-0cd4-4258-bc79-2d0c2a7798e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:34.887915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084309c-0cd4-4258-bc79-2d0c2a7798e0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.888393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084309c-0cd4-4258-bc79-2d0c2a7798e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.888883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084309c-0cd4-4258-bc79-2d0c2a7798e0 None admin] PATH_INFO: `/v3/users/21f188818fe74c5fb8aef4d7989168b0` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.890337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084309c-0cd4-4258-bc79-2d0c2a7798e0 None admin] RBAC: Authorizing `identity:get_user(user_id=21f188818fe74c5fb8aef4d7989168b0)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:34.892053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084309c-0cd4-4258-bc79-2d0c2a7798e0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:34.894347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 129/255] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/users/21f188818fe74c5fb8aef4d7989168b0 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:34.895887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:34.896370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:34.896838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:34.903558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:34.917247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:34.918877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:35.003531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 130/256] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:35.007235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.041726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022991af-4efb-40aa-9d5d-f829d95f110e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.115020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.115560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.115844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.116115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.116790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:35.117093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:35.118660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:35.134413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 127/257] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:35.136674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.147949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c6b4d-090b-40fb-a671-99915d6b6897 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.198142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:35.212859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.213625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.213940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.214323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.238628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:35.256606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0ad95-5ac1-402d-832d-0bf9540d27b2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.291590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.292477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.292930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.293347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.303907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:35.309045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:35.310174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:35.311201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:35.313107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 128/258] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:35.318285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.342190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:35.343760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:35.423433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2fc18-6049-4b01-8245-6eeb7618bf69 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.457913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.458607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.458941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.459220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] PATH_INFO: `/v3/users/a1f5a8ba16f1409ca73a2ea70c44ea9d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.460400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] RBAC: Authorizing `identity:get_user(user_id=a1f5a8ba16f1409ca73a2ea70c44ea9d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:35.461908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:35.463912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 129/259] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/users/a1f5a8ba16f1409ca73a2ea70c44ea9d => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:35.464797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.479798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:35.566448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 131/260] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:35.568332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.620169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c6258-2d59-4f13-af11-dc329f1beb06 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.654681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.655299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.655582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.655848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.656568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:35.656892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:35.658312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:35.659745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:35.671685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 130/261] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:35.672544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.748857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:35.774256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a448-7952-4338-a4a8-026718eb40d1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.831988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90792c91-b3ee-47a0-9193-2ca595326aee None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:35.836116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.837123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.838341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.838815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.841624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:35.843339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:35.848542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 131/262] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:35.849937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:35.902244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:35.904719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:35.905199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:35.906555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:35.909067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:35.911596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:35.915832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 132/263] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:35.919208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:36.049007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61db62-2ec5-4cc8-9d03-97d5258754c7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:36.118521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:36.119489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:36.119909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.120666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.121260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] PATH_INFO: `/v3/users/21f188818fe74c5fb8aef4d7989168b0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.123714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bb281-46fd-427f-9f7e-905522711486 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:36.125842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] RBAC: Authorizing `identity:get_user(user_id=21f188818fe74c5fb8aef4d7989168b0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:36.127435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:36.130487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 132/264] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/users/21f188818fe74c5fb8aef4d7989168b0 => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:36.131344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:36.156220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:36.196376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:36.198867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.199161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.199442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.202613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=21f188818fe74c5fb8aef4d7989168b0) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:36.202928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:36.204422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:36.216747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 133/265] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:35 2022] GET /identity/v3/role_assignments?user.id=21f188818fe74c5fb8aef4d7989168b0&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:36.218017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:36.312169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d881-52cc-45a7-871a-8d03cd4e9783 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:36.372877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f1f31-b5ad-4852-90fe-1b12bc5a5d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:36.373428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1f31-b5ad-4852-90fe-1b12bc5a5d59 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.373715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1f31-b5ad-4852-90fe-1b12bc5a5d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.373991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1f31-b5ad-4852-90fe-1b12bc5a5d59 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/a1f5a8ba16f1409ca73a2ea70c44ea9d/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.382891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f1f31-b5ad-4852-90fe-1b12bc5a5d59 None admin] RBAC: Authorizing `identity:create_grant(project_id=f446bfe68eb142619642d064c9ee2a41, user_id=a1f5a8ba16f1409ca73a2ea70c44ea9d, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:36.384474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f1f31-b5ad-4852-90fe-1b12bc5a5d59 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:36.413666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 133/266] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:36 2022] PUT /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/a1f5a8ba16f1409ca73a2ea70c44ea9d/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:36.415494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4471f5-e6ea-4888-9dcd-117c3c64256c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:36.434921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e9f0-04f3-4d94-9c7f-61fac4816698 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.435247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e9f0-04f3-4d94-9c7f-61fac4816698 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.435480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e9f0-04f3-4d94-9c7f-61fac4816698 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.437669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 134/267] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:27:36.442935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.443236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.443493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.463818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efd1f-9df5-4e40-95c9-f4740b95771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:36.464509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efd1f-9df5-4e40-95c9-f4740b95771d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.464800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efd1f-9df5-4e40-95c9-f4740b95771d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.465129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efd1f-9df5-4e40-95c9-f4740b95771d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.468183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:36.469668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:36.502145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:36.546761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450efd1f-9df5-4e40-95c9-f4740b95771d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:36.548430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450efd1f-9df5-4e40-95c9-f4740b95771d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:36.552560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:36.558487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 135/268] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:36.590103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:36.727199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85891a9f-5542-454e-a13b-a1b22ddbd686 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:36.730453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 134/269] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:36.730662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96440b7-0377-4347-ae7b-029d8d6b6722 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.730943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96440b7-0377-4347-ae7b-029d8d6b6722 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.731214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96440b7-0377-4347-ae7b-029d8d6b6722 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.732620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 135/270] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:36.736279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96440b7-0377-4347-ae7b-029d8d6b6722 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:36.772064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a624ac-2d2b-4ab6-a995-622c4c1a6432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:36.772720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a624ac-2d2b-4ab6-a995-622c4c1a6432 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.773001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a624ac-2d2b-4ab6-a995-622c4c1a6432 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.773276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a624ac-2d2b-4ab6-a995-622c4c1a6432 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.788186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a624ac-2d2b-4ab6-a995-622c4c1a6432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-c5a624ac-2d2b-4ab6-a995-622c4c1a6432 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:36.795526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:36.796461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:36.797277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:36.799596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 136/271] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:36.802305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.802305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.802305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.822400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:36.826939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:36.829060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:36.870432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96440b7-0377-4347-ae7b-029d8d6b6722 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:36.894784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:36.912639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:36.913398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:36.913752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:36.914099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:36.915492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 137/272] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:36.916556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:36.922193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:36.924601 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:36.926915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 136/273] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:36 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:36.933349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.042963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c2c96-879a-4152-bde7-88ea3d87d8cf None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.043158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcedfaf-b9ff-4525-9ca5-a1fd304371b3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.073855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:37.080055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.081075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.081591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.082025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.082649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.082751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.082964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.083243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.083926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:37.084236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.084483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] RBAC: Adding query filter params (role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=21f188818fe74c5fb8aef4d7989168b0) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:37.084999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.085642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.086947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.093850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 137/274] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:36 2022] GET /identity/v3/role_assignments?user.id=21f188818fe74c5fb8aef4d7989168b0&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=14f7282e89144655a81a327a65c198a5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:37.094982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.108989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 138/275] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:37.110921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.200511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:37.225035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565132fb-69e2-42c2-868b-4a576a015904 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.267577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8d041-82af-4a17-886a-db3a8805d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.268847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d041-82af-4a17-886a-db3a8805d362 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d041-82af-4a17-886a-db3a8805d362 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.269855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d041-82af-4a17-886a-db3a8805d362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.271430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8d041-82af-4a17-886a-db3a8805d362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.273191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8d041-82af-4a17-886a-db3a8805d362 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.275572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 138/276] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:37.277200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9c054-5135-481c-ab8c-32abea07cf68 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.277754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9c054-5135-481c-ab8c-32abea07cf68 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.278348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9c054-5135-481c-ab8c-32abea07cf68 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.279814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 139/277] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:37.281168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4d972-7f43-4ee1-9ae0-5428e48acd24 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.281827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9c054-5135-481c-ab8c-32abea07cf68 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.330602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1c1d-5fd8-41ec-9d0d-5ebde5d8addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.331264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1c1d-5fd8-41ec-9d0d-5ebde5d8addc None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.331543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1c1d-5fd8-41ec-9d0d-5ebde5d8addc None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.331814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1c1d-5fd8-41ec-9d0d-5ebde5d8addc None admin] PATH_INFO: `/v3/users/849f6cc764b4447da3c97e676fd0f9d2` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.350825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1c1d-5fd8-41ec-9d0d-5ebde5d8addc None admin] RBAC: Authorizing `identity:get_user(user_id=849f6cc764b4447da3c97e676fd0f9d2)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.352951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1c1d-5fd8-41ec-9d0d-5ebde5d8addc None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.354721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 139/278] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:37 2022] GET /identity/v3/users/849f6cc764b4447da3c97e676fd0f9d2 => generated 293 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:37.355916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.356204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.356533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.365012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:37.376458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:37.377664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:37.412719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9c054-5135-481c-ab8c-32abea07cf68 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.456617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.457147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.457429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.457704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.458379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.459817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.467804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 140/279] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:37.469210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.494188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 140/280] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:27:37 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:37.494188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.521841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:37.597710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09654b-1652-4ec3-a2b5-ebf0552b7eb1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.608995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0494-c341-41c8-8c72-aa904ce655a8 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.634056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:37.641851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.642540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.643292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.644050 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, user.id=849f6cc764b4447da3c97e676fd0f9d2, scope.system=all) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:37.644377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.646136 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.653596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 141/281] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 9 02:27:37 2022] GET /identity/v3/role_assignments?user.id=849f6cc764b4447da3c97e676fd0f9d2&scope.system=all&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:37.657192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.657943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.658334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.658610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.673034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:37.678743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:37.679737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:37.680694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 141/282] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:37.686188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.806548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78874d-5029-4006-bedb-02a0180cfb79 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:37.808349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:37.847833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:37.848446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:37.848722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:37.848991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:37.849667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:37.849980 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:37.851532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:37.861445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 142/283] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:37.867468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:37.974186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aab5a-67f8-4563-967d-1e971c081e94 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:38.011124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:38.011831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:38.012150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:38.012375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] PATH_INFO: `/v3/users/7d601c5dd5ed4c47ae280c3e4450728b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:38.013532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] RBAC: Authorizing `identity:get_user(user_id=7d601c5dd5ed4c47ae280c3e4450728b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:38.015091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:38.016854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 142/284] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:37 2022] GET /identity/v3/users/7d601c5dd5ed4c47ae280c3e4450728b => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:38.022301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:38.137183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f72617-f18c-483d-a74c-1668bdaf49e0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:38.172235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5623-350d-4a15-a227-202e42ae07ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:38.172880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5623-350d-4a15-a227-202e42ae07ea None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:38.173404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5623-350d-4a15-a227-202e42ae07ea None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:38.173743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5623-350d-4a15-a227-202e42ae07ea None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:38.174977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5623-350d-4a15-a227-202e42ae07ea None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:38.176499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5623-350d-4a15-a227-202e42ae07ea None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:38.178553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 143/285] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:38 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:38.184577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:38.240065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:38.292180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4a0a1-b7a3-43f4-a373-b4997694ab20 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:38.325730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466afc92-29db-44c4-8a00-5d9876dede17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:38.326432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466afc92-29db-44c4-8a00-5d9876dede17 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:38.326710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466afc92-29db-44c4-8a00-5d9876dede17 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:38.326981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466afc92-29db-44c4-8a00-5d9876dede17 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/7d601c5dd5ed4c47ae280c3e4450728b/roles/050b752dca764106b2e5b9e1b2ff57b8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:38.328872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466afc92-29db-44c4-8a00-5d9876dede17 None admin] RBAC: Authorizing `identity:create_grant(project_id=f446bfe68eb142619642d064c9ee2a41, user_id=7d601c5dd5ed4c47ae280c3e4450728b, role_id=050b752dca764106b2e5b9e1b2ff57b8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:38.330416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466afc92-29db-44c4-8a00-5d9876dede17 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:38.345800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 143/286] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:27:38 2022] PUT /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41/users/7d601c5dd5ed4c47ae280c3e4450728b/roles/050b752dca764106b2e5b9e1b2ff57b8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:38.857526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7e82d-1f9a-4b1b-b639-b9e1a1047305 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:38.857936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7e82d-1f9a-4b1b-b639-b9e1a1047305 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:38.858105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7e82d-1f9a-4b1b-b639-b9e1a1047305 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:38.859732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 144/287] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:38.865212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:38.865503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:38.865779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:38.891449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:38.892706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:38.976807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 144/288] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:39.018165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7e82d-1f9a-4b1b-b639-b9e1a1047305 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:39.050092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:39.143943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7e82d-1f9a-4b1b-b639-b9e1a1047305 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:39.185704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:39.186372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:39.186651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:39.186929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:39.203603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:39.208986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:39.209873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:39.210736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 145/289] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:39.216378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:39.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:39.358489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111a1ef-163b-4911-9b61-3d51680608b3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:39.398677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:39.399245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:39.399525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:39.399794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:39.400461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:39.400761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:39.402133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:39.411261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 145/290] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:39.419379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:39.510592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:39.519545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca108d40-e282-47d2-88df-a11c30873f54 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:39.553055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:39.554110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:39.554611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:39.555079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] PATH_INFO: `/v3/users/a1f5a8ba16f1409ca73a2ea70c44ea9d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:39.556421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] RBAC: Authorizing `identity:get_user(user_id=a1f5a8ba16f1409ca73a2ea70c44ea9d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:39.558120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:39.559939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 146/291] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:39 2022] GET /identity/v3/users/a1f5a8ba16f1409ca73a2ea70c44ea9d => generated 297 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:39.565737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:39.663672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:39.670626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc3aed-e552-4102-b3ac-46aae29bebcc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:39.714967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec449d6d-96f7-45de-b0a3-d4b269a6cab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:39.715588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec449d6d-96f7-45de-b0a3-d4b269a6cab7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:39.715867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec449d6d-96f7-45de-b0a3-d4b269a6cab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:39.716137 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec449d6d-96f7-45de-b0a3-d4b269a6cab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:39.795426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec449d6d-96f7-45de-b0a3-d4b269a6cab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:39.796885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec449d6d-96f7-45de-b0a3-d4b269a6cab7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:39.894820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:39.898424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 146/292] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:39.904382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.004930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694313b4-9972-4e2a-94c3-f632a236f825 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.008130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a8a1b-8681-493c-ab53-7702bd537cc5 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.008705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a8a1b-8681-493c-ab53-7702bd537cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.008800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a8a1b-8681-493c-ab53-7702bd537cc5 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.011232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 147/293] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:40.023944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.024411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.024779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.045044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:40.046739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:40.064366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.065011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.065289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.065561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.070825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:40.072390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:40.074052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 147/294] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:39 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:40.087470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.123704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 148/295] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:40.145797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.171509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:40.254067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58687311-4ae0-40e6-8be6-cb912a7e12bd None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.288460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:40.296258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e9db-bbb5-4143-a3e2-c1fb478083cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.296908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e9db-bbb5-4143-a3e2-c1fb478083cd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.297302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e9db-bbb5-4143-a3e2-c1fb478083cd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.297588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e9db-bbb5-4143-a3e2-c1fb478083cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.308819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e9db-bbb5-4143-a3e2-c1fb478083cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3fa8e9db-bbb5-4143-a3e2-c1fb478083cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:40.313904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:40.314831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:40.316224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 149/296] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:40.317783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001ec6-dd8a-4a11-99f7-9c1d51e42ca0 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.317783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001ec6-dd8a-4a11-99f7-9c1d51e42ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.317783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001ec6-dd8a-4a11-99f7-9c1d51e42ca0 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.318210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a26673-b1b8-4314-9e8e-dc1867961fae None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.319584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 150/297] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:40.324921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d53f6a-58ea-463d-8b33-535910938bf4 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.325201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d53f6a-58ea-463d-8b33-535910938bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.325551 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d53f6a-58ea-463d-8b33-535910938bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.344765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-00d53f6a-58ea-463d-8b33-535910938bf4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:40.345960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-00d53f6a-58ea-463d-8b33-535910938bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:40.360281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.360892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.361172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.361589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.362715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=a1f5a8ba16f1409ca73a2ea70c44ea9d) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:40.363028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:40.364912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:40.374238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 148/298] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/role_assignments?user.id=a1f5a8ba16f1409ca73a2ea70c44ea9d&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:40.375120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.433553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 151/299] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:40.434971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49cc3-bd19-4b92-9373-006a505c5297 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.435257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49cc3-bd19-4b92-9373-006a505c5297 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.435527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49cc3-bd19-4b92-9373-006a505c5297 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.438308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 152/300] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:40.441983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.442296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.442565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.465118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:40.467535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:40.510754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:40.521838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ce335-7707-4071-b1e3-11595729fe6a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.559030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.559842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.564572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.564572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.564572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:40.564572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:40.564572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:40.565326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 153/301] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:40.566890 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.574141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 149/302] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:40.579611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.635887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:40.695715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563eabd-85e0-474c-a3f8-cf87881db4fc None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.703993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa0b59-8282-40ad-b6e3-d850f3868d98 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.730524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.731266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.731651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.731923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.733102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:40.734776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:40.736499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 154/303] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:40.737090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.742470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.743156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.743431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.743720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] PATH_INFO: `/v3/users/849f6cc764b4447da3c97e676fd0f9d2` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.744851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] RBAC: Authorizing `identity:get_user(user_id=849f6cc764b4447da3c97e676fd0f9d2)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:40.746747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:40.748849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 150/304] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/users/849f6cc764b4447da3c97e676fd0f9d2 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:40.753823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.777207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:40.844113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896f83d-f468-4bd4-b51c-16fb9cab2416 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.863425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec46fbb-865a-4efc-899a-7cefd7361b12 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:40.867804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:40.886910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.887544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.887825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.888095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.898875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1de02-bc2d-4fc5-b092-700c05bc5ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:40.899037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:40.899520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1de02-bc2d-4fc5-b092-700c05bc5ddc None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.899798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1de02-bc2d-4fc5-b092-700c05bc5ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.900068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1de02-bc2d-4fc5-b092-700c05bc5ddc None admin] PATH_INFO: `/v3/groups` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.900698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1de02-bc2d-4fc5-b092-700c05bc5ddc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:40.902142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1de02-bc2d-4fc5-b092-700c05bc5ddc None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:40.904026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:40.904954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:40.905793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 155/305] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:40.906808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:40.921642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 151/306] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Aug 9 02:27:40 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:27:40.922976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57455fc5-1163-4bd3-af92-81e6ac2c9186 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:40.923291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57455fc5-1163-4bd3-af92-81e6ac2c9186 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:40.923568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57455fc5-1163-4bd3-af92-81e6ac2c9186 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:40.929257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 152/307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:40.930240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57455fc5-1163-4bd3-af92-81e6ac2c9186 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.021786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:41.024219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea570b-fe1a-47a6-8b39-5c719157afe4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.043215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57455fc5-1163-4bd3-af92-81e6ac2c9186 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.062467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab3089-e7f1-4a1b-9f5a-c64a305762d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.063192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab3089-e7f1-4a1b-9f5a-c64a305762d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.063484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab3089-e7f1-4a1b-9f5a-c64a305762d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.063752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab3089-e7f1-4a1b-9f5a-c64a305762d1 None admin] PATH_INFO: `/v3/system/users/849f6cc764b4447da3c97e676fd0f9d2/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.066073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab3089-e7f1-4a1b-9f5a-c64a305762d1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=849f6cc764b4447da3c97e676fd0f9d2, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.067746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab3089-e7f1-4a1b-9f5a-c64a305762d1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:41.078273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 156/308] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Aug 9 02:27:40 2022] PUT /identity/v3/system/users/849f6cc764b4447da3c97e676fd0f9d2/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:41.079552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.079842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.080118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.088831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.088831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.089148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.089385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.090105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:41.090859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.091934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:41.101591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:41.102843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:41.106265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 153/309] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:41.109111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.152206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:41.194210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 157/310] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:41.219321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.256233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68fecd-ef9e-446e-9381-c5f8c44d11e4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.297476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.298145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.298450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.298719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] PATH_INFO: `/v3/users/de7cce94aa154ebeac718278990cdceb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.312521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] RBAC: Authorizing `identity:get_user(user_id=de7cce94aa154ebeac718278990cdceb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.313996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:41.319681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 154/311] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:41 2022] GET /identity/v3/users/de7cce94aa154ebeac718278990cdceb => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:41.322428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.325497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:41.346665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc075b5-daaa-44f5-ad5a-a6bcf5cdf543 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.382272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.382832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.383108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.383383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.393899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:41.398774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:41.399761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:41.400979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 158/312] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:41.407042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.438277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:41.449906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114521-d961-4369-8ff5-774cdf046418 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.488693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.489429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.489800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.490137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.490973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] RBAC: Adding query filter params (role.id=050b752dca764106b2e5b9e1b2ff57b8, user.id=de7cce94aa154ebeac718278990cdceb, scope.system=all) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:41.491355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.492941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:41.505395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 155/313] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 9 02:27:41 2022] GET /identity/v3/role_assignments?user.id=de7cce94aa154ebeac718278990cdceb&scope.system=all&role.id=050b752dca764106b2e5b9e1b2ff57b8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:41.510358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aad96-fe00-41f6-9de3-4dab9e9b836a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.551937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.552584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.552860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.553127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.553935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:41.554510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.556408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:41.568503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 159/314] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:41.574374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.648272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:41.684447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24458fd1-ac2e-4cb5-873c-aff6d1c8cc10 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.718768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.719412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.719693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.719964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] PATH_INFO: `/v3/users/7d601c5dd5ed4c47ae280c3e4450728b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.721106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] RBAC: Authorizing `identity:get_user(user_id=7d601c5dd5ed4c47ae280c3e4450728b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.722974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:41.724956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 156/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:41 2022] GET /identity/v3/users/7d601c5dd5ed4c47ae280c3e4450728b => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:41.730321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:41.825937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:41.832456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd0ef4-459c-4ee1-a4f5-f8489009b787 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:41.867443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:41.868239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:41.868517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:41.868835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:41.935437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:41.936865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:42.034724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:42.042355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 160/316] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:42.047921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:42.146446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adfdc1-dc11-42e2-a127-b6a1299fff49 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:42.184107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ea3c2-7449-4aed-8340-33db32f7cec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:42.184759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea3c2-7449-4aed-8340-33db32f7cec8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:42.185041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea3c2-7449-4aed-8340-33db32f7cec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:42.185320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ea3c2-7449-4aed-8340-33db32f7cec8 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:42.186558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ea3c2-7449-4aed-8340-33db32f7cec8 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:42.188019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ea3c2-7449-4aed-8340-33db32f7cec8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:42.190068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 157/317] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:42 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:42.195838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:42.307626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d13dbf-d1c6-4695-b4d9-f1202766fd3d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:42.341989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:42.342635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:42.342919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:42.343197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:42.343930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] RBAC: Adding query filter params (role.id=050b752dca764106b2e5b9e1b2ff57b8, scope.project.id=f446bfe68eb142619642d064c9ee2a41, user.id=7d601c5dd5ed4c47ae280c3e4450728b) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:42.344241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:42.345700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7839f-d409-4617-b422-bd239a41d029 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:42.352430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 161/318] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:27:42 2022] GET /identity/v3/role_assignments?user.id=7d601c5dd5ed4c47ae280c3e4450728b&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=050b752dca764106b2e5b9e1b2ff57b8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:43.122196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8792-4f27-4874-aa13-c0da54ce5b5b None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.122748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8792-4f27-4874-aa13-c0da54ce5b5b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.122823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d8792-4f27-4874-aa13-c0da54ce5b5b None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.124298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 158/319] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:43.130189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.130623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.130747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.132056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63c9b1-504a-4a0e-91cc-4198270073bf None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.132334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63c9b1-504a-4a0e-91cc-4198270073bf None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.132606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63c9b1-504a-4a0e-91cc-4198270073bf None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.134046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 159/320] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:27:43.138707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.138996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.139273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.145364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.151485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:43.152679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:43.157940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:43.159132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:43.205886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.220471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 162/321] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:43.238194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 160/322] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:43.243078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.259775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.353807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a9bfe-fd2d-4b77-95bb-aaf8d7d597c0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.361762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29162341-79f0-4f23-bae2-7cca024cded6 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.390023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.390740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.391021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.391293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.392459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:43.393226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.394405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:43.398493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 163/323] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:43.399261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.400152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.400567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.400971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.402324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.412089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:43.417103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:43.417988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:43.419568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 161/324] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:43.424412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.468875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.513221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808659d8-3e93-455e-856b-8fddd47afb73 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.528373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9ab5d-b0e0-4673-a18b-0cf2a4b7d555 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.548716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af3f6-ca45-4679-9975-c12113144deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.549334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af3f6-ca45-4679-9975-c12113144deb None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.549612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af3f6-ca45-4679-9975-c12113144deb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.549878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af3f6-ca45-4679-9975-c12113144deb None admin] PATH_INFO: `/v3/groups` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.550529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91af3f6-ca45-4679-9975-c12113144deb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:43.551979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91af3f6-ca45-4679-9975-c12113144deb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:43.569025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.569780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.570124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.570418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.571141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:43.578238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:43.578461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 164/325] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Aug 9 02:27:43 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:27:43.578547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:43.578618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86971b60-b741-4d3f-8193-28afb1d4f0b9 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.578618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86971b60-b741-4d3f-8193-28afb1d4f0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.578618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86971b60-b741-4d3f-8193-28afb1d4f0b9 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.578618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 165/326] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:43.580677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.580940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.581210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.582816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 162/327] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:43.588788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.605302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:43.606606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:43.653381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.680919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 166/328] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:43.704567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.706328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5c884-eb24-4553-872c-6dd8ef45f954 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.748695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.749991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.749991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.749991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] PATH_INFO: `/v3/users/849f6cc764b4447da3c97e676fd0f9d2` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.751441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] RBAC: Authorizing `identity:get_user(user_id=849f6cc764b4447da3c97e676fd0f9d2)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:43.752863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:43.758188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 163/329] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/users/849f6cc764b4447da3c97e676fd0f9d2 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:43.760622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.763613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.828969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcab84-8fce-400c-9b4b-446cf4533c97 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.876506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.877480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.877765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.878038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.892145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:43.892470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e616cf-e313-44df-8338-e93a897b7522 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:43.897043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:43.897934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:43.898789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:43.899438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 167/330] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:43.906311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:43.921668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:43.941620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:43.942486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:43.942769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:43.943525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:43.944119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] RBAC: Adding query filter params (role.id=5106c6bcdc5a482eb55acc44c11b638b, user.id=849f6cc764b4447da3c97e676fd0f9d2, scope.system=all) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:43.945351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:43.945858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:43.965480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 164/331] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/role_assignments?user.id=849f6cc764b4447da3c97e676fd0f9d2&scope.system=all&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:44.046478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a2be-ebf9-4025-8963-29adaa96e1fd None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:44.055564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:44.085455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:44.086365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:44.086855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:44.087268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:44.088135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:44.088585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:44.090141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:44.101152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 168/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:44.107264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:44.232171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6aa000-3af8-4502-b0ad-5483af357d9a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:44.270661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10773476-b5f3-44a5-ac6b-9c43f911fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:44.271695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10773476-b5f3-44a5-ac6b-9c43f911fad6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:44.272152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10773476-b5f3-44a5-ac6b-9c43f911fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:44.272598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10773476-b5f3-44a5-ac6b-9c43f911fad6 None admin] PATH_INFO: `/v3/users/de7cce94aa154ebeac718278990cdceb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:44.273976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10773476-b5f3-44a5-ac6b-9c43f911fad6 None admin] RBAC: Authorizing `identity:get_user(user_id=de7cce94aa154ebeac718278990cdceb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:44.275678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10773476-b5f3-44a5-ac6b-9c43f911fad6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:44.277469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 165/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:44 2022] GET /identity/v3/users/de7cce94aa154ebeac718278990cdceb => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:44.283184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:44.402518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3351-9580-491c-bbbb-ba64ec58356c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:44.440992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969156c-4e89-4dcb-8b5d-97b628a0630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:44.441669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969156c-4e89-4dcb-8b5d-97b628a0630e None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:44.441952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969156c-4e89-4dcb-8b5d-97b628a0630e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:44.442258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969156c-4e89-4dcb-8b5d-97b628a0630e None admin] PATH_INFO: `/v3/system/users/de7cce94aa154ebeac718278990cdceb/roles/050b752dca764106b2e5b9e1b2ff57b8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:44.443701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969156c-4e89-4dcb-8b5d-97b628a0630e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=de7cce94aa154ebeac718278990cdceb, role_id=050b752dca764106b2e5b9e1b2ff57b8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:44.445089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969156c-4e89-4dcb-8b5d-97b628a0630e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:44.459609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 169/334] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Aug 9 02:27:44 2022] PUT /identity/v3/system/users/de7cce94aa154ebeac718278990cdceb/roles/050b752dca764106b2e5b9e1b2ff57b8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:46.202383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3869-a17f-4077-a64d-92ac1e024b78 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.202383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3869-a17f-4077-a64d-92ac1e024b78 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.202383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3869-a17f-4077-a64d-92ac1e024b78 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.203383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 166/335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:46.208920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.209384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.209461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.231574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:46.232790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:46.260309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:46.314276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 170/336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:46.359514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3869-a17f-4077-a64d-92ac1e024b78 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:46.429949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:46.494360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3869-a17f-4077-a64d-92ac1e024b78 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:46.531877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a96ae5-af70-44bb-824e-cd27c46c16b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:46.532529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a96ae5-af70-44bb-824e-cd27c46c16b6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.532806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a96ae5-af70-44bb-824e-cd27c46c16b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.533080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a96ae5-af70-44bb-824e-cd27c46c16b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.543853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a96ae5-af70-44bb-824e-cd27c46c16b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-31a96ae5-af70-44bb-824e-cd27c46c16b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:46.549209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:46.550119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:46.550983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 167/337] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:46.556538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:46.559835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c29512-d7b0-49be-b1b4-01c826227c17 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.560118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c29512-d7b0-49be-b1b4-01c826227c17 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.560389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c29512-d7b0-49be-b1b4-01c826227c17 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.561862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 168/338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:46.563012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6560-a368-41e1-957d-f8de92fcd0fd None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.563296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6560-a368-41e1-957d-f8de92fcd0fd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.563565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6560-a368-41e1-957d-f8de92fcd0fd None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.564739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 169/339] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:46.569604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58520a-c84e-490d-aa23-d994657d9e3c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.569888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58520a-c84e-490d-aa23-d994657d9e3c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.570187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58520a-c84e-490d-aa23-d994657d9e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.588456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-9d58520a-c84e-490d-aa23-d994657d9e3c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:46.589673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9d58520a-c84e-490d-aa23-d994657d9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:46.658363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 170/340] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:46.660410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.660960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.661433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.678537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e7d2c-7dcc-4782-8b6e-6e949cf82c86 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:46.681736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:46.683215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:46.714818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:46.724571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:46.725409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.725851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.726303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.727133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:46.727668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:46.729190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:46.738594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 171/341] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:46.739639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:46.762460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 171/342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:46.763253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:46.770591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:46.844424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda6b30-2572-467e-a512-b33b93ad9191 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:46.880620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:46.881535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.881965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.882967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:46.898642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:46.904953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:46.905845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:46.906704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:46.909065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace700b8-fb00-4139-beb3-5b1ab6f5b4b3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:46.909545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 172/343] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:46.910689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:46.983441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:46.984066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:46.984504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:46.984915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.002749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.020316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e071-a839-4e69-b910-b87197610aa5 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.047417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.057147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.058112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.058903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.059374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.072781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:47.078945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:47.079867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:47.080724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:47.083418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 173/344] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:47.084587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:47.103530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.105087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.199199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a21d4-279f-4001-89c9-85c67c295baf None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.235775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.237204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.237747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.238251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.239194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:47.239777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.241375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.251273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 174/345] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:47.252431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:47.268361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.288846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 172/346] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:47.289638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:47.360902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8398a87-96a2-4502-8d17-5e0856736386 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.398294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.404535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.405362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.410534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.410982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.411834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:47.412342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.413907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.427157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 175/347] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:47.428964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:47.448834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb157f6-0c62-4210-9824-bce8f9ef82a9 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.486731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.487588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.488029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.488439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.556858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.558584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.575404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.602675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c2205-1738-4ec9-b745-9998a02545b3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.625744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.662118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 173/348] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:47.663215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:47.667479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f8346-a078-4bdb-a230-7ef70e7d62d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.668360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f8346-a078-4bdb-a230-7ef70e7d62d1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.668786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f8346-a078-4bdb-a230-7ef70e7d62d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.669203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f8346-a078-4bdb-a230-7ef70e7d62d1 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.674985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f8346-a078-4bdb-a230-7ef70e7d62d1 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.676615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f8346-a078-4bdb-a230-7ef70e7d62d1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.678494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 176/349] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:47 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:47.679936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe01f58-90be-4363-8c74-8e9cf35a952f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.680360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe01f58-90be-4363-8c74-8e9cf35a952f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.680808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe01f58-90be-4363-8c74-8e9cf35a952f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.683333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 177/350] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:47.684215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe01f58-90be-4363-8c74-8e9cf35a952f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:47.796631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8509b46-661a-466d-bff4-36e73e655af5 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.838373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.839086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.839364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.839634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.845312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe01f58-90be-4363-8c74-8e9cf35a952f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:47.889683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e230a-1312-42aa-9a6d-e11f1551211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:47.890635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e230a-1312-42aa-9a6d-e11f1551211d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.891056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e230a-1312-42aa-9a6d-e11f1551211d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.891562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e230a-1312-42aa-9a6d-e11f1551211d None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.892038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.893203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e230a-1312-42aa-9a6d-e11f1551211d None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.894909 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e230a-1312-42aa-9a6d-e11f1551211d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.896797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 178/351] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:47 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:47.898121 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:47.898656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:47.899119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:47.911673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:47.921715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:47.923591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:47.925921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:47.927704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:48.016186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 179/352] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:48.022193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.044891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 174/353] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:48.045859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.135006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.143194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86728e-0583-487b-b396-8b860e025fd2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.172782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587e429-e359-4c78-9092-c5e26c3f9ba6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.182800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da317c6-363f-4fe4-9a66-dc7317c056c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.183399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da317c6-363f-4fe4-9a66-dc7317c056c3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.183676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da317c6-363f-4fe4-9a66-dc7317c056c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.184117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da317c6-363f-4fe4-9a66-dc7317c056c3 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.186175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.192908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da317c6-363f-4fe4-9a66-dc7317c056c3 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.194405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da317c6-363f-4fe4-9a66-dc7317c056c3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.196964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 180/354] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.197466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11855b-f155-4182-bba0-3d84b2f971d8 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.197752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11855b-f155-4182-bba0-3d84b2f971d8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.198979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11855b-f155-4182-bba0-3d84b2f971d8 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.199400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 181/355] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:48.205336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11855b-f155-4182-bba0-3d84b2f971d8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.213825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.215436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.215436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.215436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.217247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.217612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.219242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 175/356] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.220000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.313742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11855b-f155-4182-bba0-3d84b2f971d8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.351876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbc9ac-9a7d-42f9-bce7-f039f1002138 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.355177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44115f02-60c3-4821-95cd-9f3ed959a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.356276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44115f02-60c3-4821-95cd-9f3ed959a43e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.356276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44115f02-60c3-4821-95cd-9f3ed959a43e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.356276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44115f02-60c3-4821-95cd-9f3ed959a43e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.368241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44115f02-60c3-4821-95cd-9f3ed959a43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-44115f02-60c3-4821-95cd-9f3ed959a43e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:48.373103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:48.374042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:48.374913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:48.375536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 182/357] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:48.376590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f28f1-63d0-4720-95fe-927600dfcf1c None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.376871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f28f1-63d0-4720-95fe-927600dfcf1c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.377140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f28f1-63d0-4720-95fe-927600dfcf1c None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.378457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 183/358] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:48.379293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f28f1-63d0-4720-95fe-927600dfcf1c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.394476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638a19-0d7e-49a7-801c-5a98d94dfd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.394476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638a19-0d7e-49a7-801c-5a98d94dfd77 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.394476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638a19-0d7e-49a7-801c-5a98d94dfd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.394817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638a19-0d7e-49a7-801c-5a98d94dfd77 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.395756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638a19-0d7e-49a7-801c-5a98d94dfd77 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.397204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638a19-0d7e-49a7-801c-5a98d94dfd77 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.398904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 176/359] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.400070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.400352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.400625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.411901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.426225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:48.426615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:48.451203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.501677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 177/360] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:48.504833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.504951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f28f1-63d0-4720-95fe-927600dfcf1c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.547326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.548314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.549126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.549408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.550175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=32b9c55705c240fa9811f2211ec09562) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:48.550513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.552447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.558220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 184/361] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.563260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.625878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d34fab-f8be-4ab0-8f1b-e906d8e81296 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.679928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.680570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.680870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.681136 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.681934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=f446bfe68eb142619642d064c9ee2a41) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:48.682288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.684020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcd4f4-592f-4c8f-9c58-788cad81a277 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.686396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.691019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 178/362] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.692644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.693185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.693618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.694995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5556f42-f1b8-4e62-99ff-c5e5f90c07a8 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.714279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:48.715760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:48.734553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.748051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.749122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.749599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.750072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.751246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:48.751619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.753198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.763826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 185/363] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.765015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.812198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 179/364] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:48.813177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.911673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08235d8b-7153-4152-a022-da734c408f2d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.947581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.948446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.948614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76721a2a-b439-4d64-9107-51aafa397a9f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:48.948895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.949371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:48.951511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:48.953290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:48.956611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 186/365] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:48.957640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:48.981082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:48.991106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:48.991762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:48.992167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:48.992433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:49.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:49.014339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 02:27:49.015199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:49.019349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 180/366] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:49.020472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.060942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:49.121712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ac8fc-9cdf-418f-b14e-74b3cc5e6bab None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.144323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94856e-5ae2-4892-872a-537b78af6e8e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.158447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.159514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.159997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.160469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.180249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.180943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.181220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.181488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.192555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:49.198785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:49.199689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 181/367] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:49.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.246741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:49.248377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:49.258104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:49.306611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:49.326169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfa729-3ec3-41c7-bca2-8cebb91d40ef None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.369914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 187/368] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:49.370841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.375527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.376148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.376430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.376685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.377435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=32b9c55705c240fa9811f2211ec09562) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:49.377726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:49.379231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:49.389466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 182/369] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=14f7282e89144655a81a327a65c198a5 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:49.391024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.550336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a4f3e-6048-4c04-832f-cfa8face55e2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.562434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38a991-85db-417f-9c65-06a077064a40 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.565191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:49.589458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.590096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.590407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.590712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.591368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:49.591672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:49.593061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:49.602662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 183/370] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:49.603523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.618348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.618977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.619257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.619530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.620210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:49.620514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:49.621893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:49.632939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 188/371] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:49.634331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.654619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:49.717080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c3303-3a19-4748-ba39-fe910f441053 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.757005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.757775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.758147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.758475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.760893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:49.763866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:49.765556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 184/372] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:49.766356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:49.825042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309969b-1c8a-47c1-ae2f-8d9d6eda4e20 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.854838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:49.888894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.889530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.889812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.890086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.935150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7416-a4bf-4935-a515-6e5ca3f45ac6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:49.977495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:49.978182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:49.978469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:49.978736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] PATH_INFO: `/v3/users/de7cce94aa154ebeac718278990cdceb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:49.979932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] RBAC: Authorizing `identity:get_user(user_id=de7cce94aa154ebeac718278990cdceb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:49.981401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:49.983336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 185/373] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/users/de7cce94aa154ebeac718278990cdceb => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:49.983982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:50.032231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.033705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.056783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:50.101705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfda21-51de-4d50-a8e2-c135dc01fcc0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:50.112989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:50.137283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:50.137937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:50.138252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:50.138525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:50.139646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.141095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.143175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 186/374] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:50.143950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:50.241023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 189/375] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:50.241893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:50.248351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374a2fc-61fc-471d-b2a2-9914b8cfb254 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:50.285271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:50.285896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:50.286193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:50.286473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:50.287214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] RBAC: Adding query filter params (role.id=050b752dca764106b2e5b9e1b2ff57b8, user.id=de7cce94aa154ebeac718278990cdceb, scope.system=all) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:50.287522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.288969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.303954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 187/376] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 9 02:27:50 2022] GET /identity/v3/role_assignments?user.id=de7cce94aa154ebeac718278990cdceb&scope.system=all&role.id=050b752dca764106b2e5b9e1b2ff57b8 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:50.304812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:50.344818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:50.420944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c759a9-ed00-46b4-83d9-98e1a2843548 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:50.421607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd68550-b20c-462f-8269-6cc7093cf9f0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:50.438226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:50.460376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:50.461147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:50.461533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:50.461893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] PATH_INFO: `/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:50.462624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:50.463092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=aaeb91952f2c4c9f8cf8e3a10b5518c9)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.463274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:50.463529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:50.463801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:50.464549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=f446bfe68eb142619642d064c9ee2a41) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:50.464712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.464869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.466785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 188/377] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:50 2022] GET /identity/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:50.468574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.475298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:50.477159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 190/378] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:50 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=14f7282e89144655a81a327a65c198a5 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:50.594828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be80a7b-92d5-4ef0-9b58-b4f50c659ee1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:50.598042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:50.653664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fb2fa-03f5-4fa4-89c4-4f59c2d30ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:50.654366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb2fa-03f5-4fa4-89c4-4f59c2d30ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:50.654646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb2fa-03f5-4fa4-89c4-4f59c2d30ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:50.654917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fb2fa-03f5-4fa4-89c4-4f59c2d30ae0 None admin] PATH_INFO: `/v3/groups/faa0cec421824a83b5b0aa14ee7e4ba0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:50.667634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fb2fa-03f5-4fa4-89c4-4f59c2d30ae0 None admin] RBAC: Authorizing `identity:get_group(group_id=faa0cec421824a83b5b0aa14ee7e4ba0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.669062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fb2fa-03f5-4fa4-89c4-4f59c2d30ae0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.671212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 189/379] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:50 2022] GET /identity/v3/groups/faa0cec421824a83b5b0aa14ee7e4ba0 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:50.677008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:50.795056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bc2f7-103f-4423-bcd2-54e8524fb461 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:50.829967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:50.830623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:50.830905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:50.831178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:50.831917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] RBAC: Adding query filter params (group.id=faa0cec421824a83b5b0aa14ee7e4ba0, role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=aaeb91952f2c4c9f8cf8e3a10b5518c9) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:50.832224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:50.833670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f888b2b-6eb4-4b18-b821-1a6bf2a09d6a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:50.840162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 191/380] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:50 2022] GET /identity/v3/role_assignments?group.id=faa0cec421824a83b5b0aa14ee7e4ba0&scope.project.id=aaeb91952f2c4c9f8cf8e3a10b5518c9&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:51.191684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dea77-3004-42a7-9ac3-4cda0a556dc5 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.192064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dea77-3004-42a7-9ac3-4cda0a556dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.192259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dea77-3004-42a7-9ac3-4cda0a556dc5 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.193633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 190/381] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:51.201691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eae765-35d5-4834-8919-b845432b9f77 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.201987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eae765-35d5-4834-8919-b845432b9f77 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.202290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eae765-35d5-4834-8919-b845432b9f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.221484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b0eae765-35d5-4834-8919-b845432b9f77 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:51.223117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b0eae765-35d5-4834-8919-b845432b9f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:51.268811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:51.307531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 192/382] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:51.341694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dea77-3004-42a7-9ac3-4cda0a556dc5 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:51.419387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a15cc4-76d0-4993-9242-ae17b4042409 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.419676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a15cc4-76d0-4993-9242-ae17b4042409 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.419952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a15cc4-76d0-4993-9242-ae17b4042409 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.421547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 193/383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:51.426760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.427339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.427339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.466783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:51.467941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:51.484545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dea77-3004-42a7-9ac3-4cda0a556dc5 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:51.544782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:51.545429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.545709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.545978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.570965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:51.576397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:51.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:51.578105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:51.590146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 191/384] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:51.592497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:51.607396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 194/385] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:51.630937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:51.643111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:51.779499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3668885-fcce-46e6-aced-137cc7604e02 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:51.821288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812d4a2-c887-44f1-8c93-73842f2d5457 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:51.823642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:51.844454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:51.845081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.845360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.845633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.846427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:51.846741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:51.848615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:51.864099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 192/386] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:51.870770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:51.903826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:51.904493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:51.904775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:51.905050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:51.935133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:51.940042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:51.940954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:51.941786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:51.943653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 195/387] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:51.964149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:51.978795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:51.990348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5413f0-539c-4078-9cd0-b8ae446cce0e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.040003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.040627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.040937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.041631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.042451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.043897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.050175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 193/388] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:51 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:52.063198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.090024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8116-5099-4054-81d4-7e5b2b5960a0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.133781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.134783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.135255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.135748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.136742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:52.137235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.138833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.147625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 196/389] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:52.159143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.178482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98618336-15b0-41c7-b66f-ddaa429453da None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.197651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:52.217573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.218541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.219018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.219531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.220858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.222519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.230328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 194/390] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:52 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:52.237789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.279946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a4ea-02cf-4853-a869-4423d3022962 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.306613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:52.316980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85230ca-96aa-47ed-b5c3-fcb6fb04a844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.317623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85230ca-96aa-47ed-b5c3-fcb6fb04a844 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.317900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85230ca-96aa-47ed-b5c3-fcb6fb04a844 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.318188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85230ca-96aa-47ed-b5c3-fcb6fb04a844 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.319423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85230ca-96aa-47ed-b5c3-fcb6fb04a844 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.320848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85230ca-96aa-47ed-b5c3-fcb6fb04a844 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.322822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 197/391] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:52 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:52.326201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f273e-3a9a-48c9-80a0-f3932602333b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.326201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f273e-3a9a-48c9-80a0-f3932602333b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.326201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f273e-3a9a-48c9-80a0-f3932602333b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.326201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 198/392] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:52.330224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f273e-3a9a-48c9-80a0-f3932602333b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.402347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aef9c-227e-4f1a-8ca5-8e88b5f120d6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.439563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36240a-6dac-4aa0-8a76-07f0361274f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.440399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36240a-6dac-4aa0-8a76-07f0361274f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.440557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:52.440997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36240a-6dac-4aa0-8a76-07f0361274f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.441208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36240a-6dac-4aa0-8a76-07f0361274f5 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562/groups/6dfab0348533470e850940a5e7cffd94/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.443324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36240a-6dac-4aa0-8a76-07f0361274f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b9c55705c240fa9811f2211ec09562, group_id=6dfab0348533470e850940a5e7cffd94, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.444884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36240a-6dac-4aa0-8a76-07f0361274f5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.452477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f273e-3a9a-48c9-80a0-f3932602333b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.462231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 195/393] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 9 02:27:52 2022] PUT /identity/v3/projects/32b9c55705c240fa9811f2211ec09562/groups/6dfab0348533470e850940a5e7cffd94/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:52.463982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.464567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.465094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.488540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:52.491141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:52.493174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.493975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.494498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.494922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.496654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.497823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.499981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 199/394] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:52 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:52.515318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.571594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 196/395] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:52.622966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.642383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:52.665313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcad90-6185-45a5-93f9-8a571db6b702 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.700447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.701126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.701403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.701674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41/groups/6dfab0348533470e850940a5e7cffd94/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.704302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] RBAC: Authorizing `identity:create_grant(project_id=f446bfe68eb142619642d064c9ee2a41, group_id=6dfab0348533470e850940a5e7cffd94, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:52.705397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:52.721695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 200/396] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 9 02:27:52 2022] PUT /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41/groups/6dfab0348533470e850940a5e7cffd94/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:27:52.750831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bd3a7-bcf9-4cc9-b201-98425e094566 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:52.797622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:52.798332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:52.798782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:52.799061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:52.813208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:52.818504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:52.819444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:52.820280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 197/397] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:52.827796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:52.881904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:52.974901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772c686-821b-4eac-a0d6-60dcb6695808 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:53.011848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:53.012474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.012757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.013030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.013717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:53.014084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:53.015570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fab42-2eb9-4fd8-8efe-032fd9bb132f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:53.024447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 201/398] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:53.030468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.032723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c2e9-4151-43aa-86a7-1f05bbcd7626 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.033010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c2e9-4151-43aa-86a7-1f05bbcd7626 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.033286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c2e9-4151-43aa-86a7-1f05bbcd7626 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.038185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 202/399] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:53.041684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.041975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.042288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.081640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:53.082824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:53.144000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:53.168735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238de7-5110-44d6-b0b7-42100594ff07 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:53.206701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:53.207634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.208059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.208476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.209887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:53.211760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:53.213949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 198/400] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:53 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:53.221120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.238179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 203/401] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:53.291651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.332077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:53.352408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac7851-c22d-463a-9345-5ab05214770d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:53.388657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:53.390343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.390938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.392384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.393886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:53.396291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:53.401654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 199/402] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:53 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:53.407602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.438565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:53.509369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617140ee-3e69-43cb-b63b-d8829be86f35 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:53.539289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f712703-2b01-4fab-8b5a-02a8d11f5dea None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:53.561104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84be75-eed2-4eef-ba50-521fc8068688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:53.562025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84be75-eed2-4eef-ba50-521fc8068688 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.562339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84be75-eed2-4eef-ba50-521fc8068688 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.562617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84be75-eed2-4eef-ba50-521fc8068688 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.577776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84be75-eed2-4eef-ba50-521fc8068688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:53.577926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30585f-934d-4834-bee2-3d322edde682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:53.578678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30585f-934d-4834-bee2-3d322edde682 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.578962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30585f-934d-4834-bee2-3d322edde682 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.579235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30585f-934d-4834-bee2-3d322edde682 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562/groups/6dfab0348533470e850940a5e7cffd94/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.581489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30585f-934d-4834-bee2-3d322edde682 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b9c55705c240fa9811f2211ec09562, group_id=6dfab0348533470e850940a5e7cffd94, role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-1d84be75-eed2-4eef-ba50-521fc8068688 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:53.582955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:53.584017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:53.585106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:53.586362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30585f-934d-4834-bee2-3d322edde682 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:53.586486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 204/403] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:53.586486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2f42-3b26-4886-9201-10304ba1cdc4 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.586797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2f42-3b26-4886-9201-10304ba1cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.590176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2f42-3b26-4886-9201-10304ba1cdc4 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.593087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 205/404] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:53.594614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2f42-3b26-4886-9201-10304ba1cdc4 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.609463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 200/405] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 9 02:27:53 2022] PUT /identity/v3/projects/32b9c55705c240fa9811f2211ec09562/groups/6dfab0348533470e850940a5e7cffd94/roles/14f7282e89144655a81a327a65c198a5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:53.611569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.612353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.612991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.637175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:53.639141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:53.722614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:53.785480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 201/406] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:53.786104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2f42-3b26-4886-9201-10304ba1cdc4 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:53.808766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.811047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:53.825887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:53.826540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:53.826916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:53.827194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:53.827973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:53.828281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:53.829658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:53.839790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 206/407] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:53.846059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:53.960356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e46751-74ed-4219-a640-533242ef1c6c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.006170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a208681-97ac-4e05-b10a-01ce7f32335e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.031765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.032429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.032709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.032982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.034175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.035630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.037243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 207/408] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:53 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:54.045888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:54.048869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:54.064981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.066029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.066553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.067018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.090187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:54.095577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:54.096456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:54.097278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:54.100495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 202/409] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:54.106408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:54.168302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037ce07-0f7b-45e0-a95c-45e8adc028ca None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.202568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:54.231312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.231957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.232239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.232516 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.236460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.237934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.244493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 208/410] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:54 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:54.245287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:54.272834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18970de7-77ea-4302-8771-d9cef00b93fe None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.334397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.335024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.335305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.335578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.336254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:54.336558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.338817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.353760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 203/411] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:54.359397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:54.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:54.468507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ccb01-5883-4d34-b201-098e9f1bbaa5 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.502868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c8c31-413f-4df1-8c57-a83e551e3391 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.514515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:54.525953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.527979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.528720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.530381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] PATH_INFO: `/v3/groups/faa0cec421824a83b5b0aa14ee7e4ba0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.532067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] RBAC: Authorizing `identity:get_group(group_id=faa0cec421824a83b5b0aa14ee7e4ba0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.534793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.539364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 204/412] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:54 2022] GET /identity/v3/groups/faa0cec421824a83b5b0aa14ee7e4ba0 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:54.545123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:54.575163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.576568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.577063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.577693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41/groups/6dfab0348533470e850940a5e7cffd94/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.582702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] RBAC: Authorizing `identity:create_grant(project_id=f446bfe68eb142619642d064c9ee2a41, group_id=6dfab0348533470e850940a5e7cffd94, role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.586602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.609247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 209/413] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 9 02:27:54 2022] PUT /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41/groups/6dfab0348533470e850940a5e7cffd94/roles/14f7282e89144655a81a327a65c198a5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:54.715381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba61d1b-81ae-4b56-8ee8-73397b8a7137 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.761373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dfe87-a14b-4cc9-8bb1-9f6f634a3330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.762026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dfe87-a14b-4cc9-8bb1-9f6f634a3330 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.762347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dfe87-a14b-4cc9-8bb1-9f6f634a3330 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.762622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dfe87-a14b-4cc9-8bb1-9f6f634a3330 None admin] PATH_INFO: `/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.766938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dfe87-a14b-4cc9-8bb1-9f6f634a3330 None admin] RBAC: Authorizing `identity:get_project(project_id=aaeb91952f2c4c9f8cf8e3a10b5518c9)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.768444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dfe87-a14b-4cc9-8bb1-9f6f634a3330 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.770532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 205/414] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:54 2022] GET /identity/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9 => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:54.776573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:54.885829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b5194-f64f-4f14-b002-d45c7411053f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:54.916826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:54.924019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36c70f-ad32-4b93-9664-b4b675172378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:54.924711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36c70f-ad32-4b93-9664-b4b675172378 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:54.924995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36c70f-ad32-4b93-9664-b4b675172378 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:54.925271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36c70f-ad32-4b93-9664-b4b675172378 None admin] PATH_INFO: `/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9/groups/faa0cec421824a83b5b0aa14ee7e4ba0/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:54.927407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36c70f-ad32-4b93-9664-b4b675172378 None admin] RBAC: Authorizing `identity:create_grant(project_id=aaeb91952f2c4c9f8cf8e3a10b5518c9, group_id=faa0cec421824a83b5b0aa14ee7e4ba0, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:54.929049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36c70f-ad32-4b93-9664-b4b675172378 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:54.944903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 210/415] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 9 02:27:54 2022] PUT /identity/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9/groups/faa0cec421824a83b5b0aa14ee7e4ba0/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:27:55.086498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b575d-5c08-4c9a-b26d-444efe86e281 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.086995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b575d-5c08-4c9a-b26d-444efe86e281 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.087367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b575d-5c08-4c9a-b26d-444efe86e281 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.089244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 206/416] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:55.094794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.095086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.095363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.121104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:55.122408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:55.158022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdecb9-db67-4b37-b2cd-a8f59be78148 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.158231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdecb9-db67-4b37-b2cd-a8f59be78148 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.158505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdecb9-db67-4b37-b2cd-a8f59be78148 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.159863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 207/417] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:27:55.165340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.165622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.165898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.187369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:55.188740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:55.200748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 211/418] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:55.231487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:55.262520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:55.284627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 208/419] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 02:27:55.319541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:55.379692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce924-82c8-4eed-bb33-2b16e5c34257 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:55.419129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:55.433582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c588-d41e-41fd-9234-f3acaae73837 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:55.448331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:55.449104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.449390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.449669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.472614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:55.479557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:55.480540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.481039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.481548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:55.483338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:55.484285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:55.485221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:55.487466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 212/420] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:55.494581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:55.502651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:55.512319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:55.513227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 02:27:55.514092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:27:55.518921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 209/421] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:27:55.524311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:55.618589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:55.685046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120cea-5f67-4672-8893-9ee136483386 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:55.704429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e21616-f300-4432-92ab-e8fd4b93322f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:55.750582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:55.751578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.752080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.752623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.753552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] RBAC: Adding query filter params (name=member) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:55.754078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:55.755715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:55.764958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 213/422] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:55.770417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:55.771057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:55.774006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.775513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.781736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.782791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:55.783336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:55.784962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:55.800449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 210/423] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:27:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:55.806077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:55.855930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:55.932143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9da4f9-b581-4e6b-9a5f-c05b77928020 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:55.959595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b0b32-829a-485f-a7ee-a12f98b65f87 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:55.973728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:55.978344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463291d3-695f-4c2a-ba71-76cc9ed24be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:55.979037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463291d3-695f-4c2a-ba71-76cc9ed24be4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:55.979719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463291d3-695f-4c2a-ba71-76cc9ed24be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:55.980259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463291d3-695f-4c2a-ba71-76cc9ed24be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:55.999284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.000384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.001294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.002020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.073108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463291d3-695f-4c2a-ba71-76cc9ed24be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.074796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463291d3-695f-4c2a-ba71-76cc9ed24be4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.081476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.082982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.121803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:56.184314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 214/424] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 02:27:56.185659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5fa42-8e07-49a6-81ad-725cf2e23a64 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.185947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5fa42-8e07-49a6-81ad-725cf2e23a64 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.186236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5fa42-8e07-49a6-81ad-725cf2e23a64 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.187869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 215/425] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:56.190507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5fa42-8e07-49a6-81ad-725cf2e23a64 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.190994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 211/426] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:56.192321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.214652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:56.300802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76708343-1d23-4fa2-a479-5625ef7e1746 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.307024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5fa42-8e07-49a6-81ad-725cf2e23a64 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.337445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43148d0-b3b3-4c2b-bad4-c9102ce4799d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.338304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43148d0-b3b3-4c2b-bad4-c9102ce4799d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.338711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43148d0-b3b3-4c2b-bad4-c9102ce4799d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.339045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43148d0-b3b3-4c2b-bad4-c9102ce4799d None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.340343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43148d0-b3b3-4c2b-bad4-c9102ce4799d None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.341955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43148d0-b3b3-4c2b-bad4-c9102ce4799d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.343213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.343739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 212/427] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:56.344205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.344489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.344767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.344865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.345113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.345389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.345901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.347441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.354186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 216/428] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:56.355042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.368763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:56.371892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:56.373110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:56.444303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 213/429] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:56.445167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.462972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f296b4-6be7-4a2a-911d-10a1da3645c0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.471620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:56.506692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.507349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.507632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.507903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.509250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.510793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.512845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 217/430] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:56.513513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.592643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:56.597459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d2e26-94da-45a3-b444-212caaa8dd6c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.614827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d390-1c51-440b-935c-0487465c9ce1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.636454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.637878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.638441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.639016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.640389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.642253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.644601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 214/431] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:56.645641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.654194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.654853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.655139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.655427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.666168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:56.671474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:56.672386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:56.673205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 218/432] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:56.673887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.751681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:56.783520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96351a1-b790-48e4-9b01-7898ef4e0138 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.799821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb044fca-f544-4fce-a8e8-162fc0895dff None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.823690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.824692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.825170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.825657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.826631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=f446bfe68eb142619642d064c9ee2a41) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:56.827141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.828776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.835333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.835971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 215/433] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:56.837027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:56.841502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.841689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.841907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.842667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=5106c6bcdc5a482eb55acc44c11b638b, scope.project.id=32b9c55705c240fa9811f2211ec09562) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:56.842992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:56.844481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b49f4-deb0-4435-a386-e4ce0760d2e2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:56.860147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 219/434] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=5106c6bcdc5a482eb55acc44c11b638b => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:56.959223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875bf5e-f1bd-42a0-a8fb-5c4d953f6675 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:56.996184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4626-0507-4eda-b9cf-6677414b7667 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.996582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4626-0507-4eda-b9cf-6677414b7667 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:56.996749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4626-0507-4eda-b9cf-6677414b7667 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:56.999005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:56.999630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:56.999911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.000185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.000874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:57.001180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:57.002783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 220/435] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:57.002902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:57.005952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.006587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.007058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.012728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 216/436] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:57.018417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.022448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.026144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.032427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:57.034020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:57.151249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 221/437] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:57.175008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.208301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd83d0b-efff-4e9d-97de-bcfc64bfe4ef None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.253091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd83e-aaad-49da-a319-fb903f98ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.253635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd83e-aaad-49da-a319-fb903f98ab56 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.253912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd83e-aaad-49da-a319-fb903f98ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.254235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd83e-aaad-49da-a319-fb903f98ab56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.297415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3467c-d879-4d01-aab4-fcc3dfe6c97a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.329617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.335529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd83e-aaad-49da-a319-fb903f98ab56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:57.335709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a07a-18b5-4875-8c8d-b9599caa6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.336351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a07a-18b5-4875-8c8d-b9599caa6e29 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.336628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a07a-18b5-4875-8c8d-b9599caa6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.336886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd83e-aaad-49da-a319-fb903f98ab56 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:57.336978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a07a-18b5-4875-8c8d-b9599caa6e29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.339966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.351538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54a07a-18b5-4875-8c8d-b9599caa6e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-dd54a07a-18b5-4875-8c8d-b9599caa6e29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:57.357452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:57.358596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:57.359733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 222/438] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:57.360556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db77b-d732-4468-bb57-d4257798e4d2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.360556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db77b-d732-4468-bb57-d4257798e4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.360556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db77b-d732-4468-bb57-d4257798e4d2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.361505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 223/439] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:27:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:27:57.364999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732db77b-d732-4468-bb57-d4257798e4d2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.455097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 217/440] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 02:27:57.459946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.459946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.459946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.474252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:27:57.475432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:27:57.476328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732db77b-d732-4468-bb57-d4257798e4d2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.510715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.511272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.511545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.511816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.512490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:57.512794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:57.514177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:57.522527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 224/441] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:57.523342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.548478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 218/442] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:27:57.549333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.584518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.592969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.642439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a91bc-97b9-4819-a66c-065cd8e20cad None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.658369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef802c-c5f5-4567-87f4-e3d8b75e5de0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.674869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.675547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.675824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.676098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] PATH_INFO: `/v3/projects/32b9c55705c240fa9811f2211ec09562` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.677251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] RBAC: Authorizing `identity:get_project(project_id=32b9c55705c240fa9811f2211ec09562)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:57.678717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:57.680563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 225/443] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/projects/32b9c55705c240fa9811f2211ec09562 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:57.681223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.695701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.696358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.696636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.696908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.763236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:57.764741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:57.777037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e7b97-9c96-4ede-a383-20127f7601ec None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.796124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.800063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:57.812938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.813568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.813846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.814119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.824412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:27:57.829510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:27:57.830479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:27:57.831302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 226/444] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:27:57.831989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.868601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 219/445] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:57.869448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:57.941212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9428c1-8e38-4e61-9d8b-83e704c0f4b5 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.976003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb971-1041-4efb-8b3c-2e800ad46767 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:57.979825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:57.980469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:57.980755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:57.981031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:57.982191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:57.983638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:57.985250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 227/446] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:57.986005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.011502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.012150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.012400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.012709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.013422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:58.013685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.015101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.016393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.023835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 220/447] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:58.024661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.037206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.087149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57164c1-6c9f-4f30-9e2d-03f60d0f18ac None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.120127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.120755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.121058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.121325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] PATH_INFO: `/v3/projects/f446bfe68eb142619642d064c9ee2a41` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.122574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] RBAC: Authorizing `identity:get_project(project_id=f446bfe68eb142619642d064c9ee2a41)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.124033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.125866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 228/448] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:57 2022] GET /identity/v3/projects/f446bfe68eb142619642d064c9ee2a41 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:58.125981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a3630-1552-41ab-8de5-6bd41f42cea7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.126709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.161788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.162434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.162720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.162994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.163725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=32b9c55705c240fa9811f2211ec09562) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:58.164059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.165626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.171433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 221/449] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=32b9c55705c240fa9811f2211ec09562&role.id=14f7282e89144655a81a327a65c198a5 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:58.172123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.233227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a341d9-5982-4766-8a4a-7b985c4a1e94 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.248961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.252875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.272803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.273688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.274286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.274698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.294698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f0ef-f8cb-4f01-bb11-6657ba61d213 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.330002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.330688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.330974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.331741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] PATH_INFO: `/v3/groups/6dfab0348533470e850940a5e7cffd94` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.332393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] RBAC: Authorizing `identity:get_group(group_id=6dfab0348533470e850940a5e7cffd94)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.333867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.335751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 222/450] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/groups/6dfab0348533470e850940a5e7cffd94 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:58.341693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.354747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.356220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.465275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec339d-9645-4302-95bf-09d5c29b7e12 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.466116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 229/451] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:27:58.471840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.491264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.497522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.508215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.508852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.509125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.509395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.510128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] RBAC: Adding query filter params (group.id=6dfab0348533470e850940a5e7cffd94, role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=f446bfe68eb142619642d064c9ee2a41) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:58.512407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.513855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.520050 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 223/452] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/role_assignments?group.id=6dfab0348533470e850940a5e7cffd94&scope.project.id=f446bfe68eb142619642d064c9ee2a41&role.id=14f7282e89144655a81a327a65c198a5 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:58.583900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a1d1-5c3d-4472-bbf9-dadfdda82a9a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.623463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b960659-6134-4292-a282-c35d200b089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.625946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b960659-6134-4292-a282-c35d200b089c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.625946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b960659-6134-4292-a282-c35d200b089c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.625946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b960659-6134-4292-a282-c35d200b089c None admin] PATH_INFO: `/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.625946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b960659-6134-4292-a282-c35d200b089c None admin] RBAC: Authorizing `identity:get_project(project_id=aaeb91952f2c4c9f8cf8e3a10b5518c9)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.627406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b960659-6134-4292-a282-c35d200b089c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.629103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 230/453] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/projects/aaeb91952f2c4c9f8cf8e3a10b5518c9 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:27:58.635161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.748348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e02121-8819-4bcc-8f1e-6f316922bf46 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.782435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5dfe5-9a12-4924-b14d-9bd9968b09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.783085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5dfe5-9a12-4924-b14d-9bd9968b09e6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.783366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5dfe5-9a12-4924-b14d-9bd9968b09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.783641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5dfe5-9a12-4924-b14d-9bd9968b09e6 None admin] PATH_INFO: `/v3/groups/faa0cec421824a83b5b0aa14ee7e4ba0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.784767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5dfe5-9a12-4924-b14d-9bd9968b09e6 None admin] RBAC: Authorizing `identity:get_group(group_id=faa0cec421824a83b5b0aa14ee7e4ba0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.786256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5dfe5-9a12-4924-b14d-9bd9968b09e6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.787851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 224/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/groups/faa0cec421824a83b5b0aa14ee7e4ba0 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:27:58.793809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b960659-6134-4292-a282-c35d200b089c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:27:58.878118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:27:58.892812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b960659-6134-4292-a282-c35d200b089c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:27:58.924667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:27:58.925274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:27:58.925554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:27:58.925827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:27:58.926590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] RBAC: Adding query filter params (group.id=faa0cec421824a83b5b0aa14ee7e4ba0, role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=aaeb91952f2c4c9f8cf8e3a10b5518c9) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:27:58.926898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:27:58.928332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a3147-c8af-421a-850a-65c5a88e5521 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:27:58.933918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 231/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 9 02:27:58 2022] GET /identity/v3/role_assignments?group.id=faa0cec421824a83b5b0aa14ee7e4ba0&scope.project.id=aaeb91952f2c4c9f8cf8e3a10b5518c9&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:01.341814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c6d17-6fdd-475f-be34-478c2b022a43 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.342383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c6d17-6fdd-475f-be34-478c2b022a43 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.342501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c6d17-6fdd-475f-be34-478c2b022a43 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.345358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 225/456] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:01.354783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8df20-604c-45c8-99b9-6a36ad435b62 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.355274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8df20-604c-45c8-99b9-6a36ad435b62 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.355351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8df20-604c-45c8-99b9-6a36ad435b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.378036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-9ed8df20-604c-45c8-99b9-6a36ad435b62 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:01.379256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9ed8df20-604c-45c8-99b9-6a36ad435b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:01.462956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 232/457] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:01.512264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c6d17-6fdd-475f-be34-478c2b022a43 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:01.584360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:01.642501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c6d17-6fdd-475f-be34-478c2b022a43 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:01.692300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:01.693315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.693792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.694289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.708835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:01.710643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-37070e9e-dbcc-4f18-bf31-69a202c4f27d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:01.729411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:01.730441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:01.731352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:01.731352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:01.731352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:01.733165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 226/458] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:01.738175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844b0ce-cdca-4b79-b465-171071cd58b4 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.738175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844b0ce-cdca-4b79-b465-171071cd58b4 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.738175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844b0ce-cdca-4b79-b465-171071cd58b4 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.738175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 233/459] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:01.741090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844b0ce-cdca-4b79-b465-171071cd58b4 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:01.741435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f68d2f-6cca-4ea2-9ec8-2bbdf1401bf0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.741724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f68d2f-6cca-4ea2-9ec8-2bbdf1401bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.741998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f68d2f-6cca-4ea2-9ec8-2bbdf1401bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.770672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-00f68d2f-6cca-4ea2-9ec8-2bbdf1401bf0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:01.770672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-00f68d2f-6cca-4ea2-9ec8-2bbdf1401bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:01.847342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:01.848964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 227/460] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:01.851139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653caf4a-067a-42d5-b14d-8240703ddc0d None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.851597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653caf4a-067a-42d5-b14d-8240703ddc0d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.852014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653caf4a-067a-42d5-b14d-8240703ddc0d None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.854917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 228/461] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:01.862034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.862565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.863047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.884356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844b0ce-cdca-4b79-b465-171071cd58b4 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:01.889414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:01.890751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:01.925027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:01.925968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:01.926474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:01.927054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:01.927879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:01.928674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:01.929125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:01.930768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:01.944277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 234/462] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:01.945634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:01.969335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 229/463] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:01.970634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.049145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95982-5a2e-464d-9da4-2adcfa756e66 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.083180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.083809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.084086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.084363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.093369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:02.095517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.096312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7802fa-ae2a-4301-b33f-9df7568337e9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.097074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:02.115190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:02.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:02.117849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:02.117849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:02.117849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:02.117849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 235/464] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:02.117849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.131219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11718cc-60a3-4438-9097-cd96b5e20040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.131960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11718cc-60a3-4438-9097-cd96b5e20040 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.132239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11718cc-60a3-4438-9097-cd96b5e20040 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.132616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11718cc-60a3-4438-9097-cd96b5e20040 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.133261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11718cc-60a3-4438-9097-cd96b5e20040 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.135195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11718cc-60a3-4438-9097-cd96b5e20040 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.158488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 230/465] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:28:01 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:02.159819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b4588-af7d-4767-b710-f7b56f7abb27 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.160112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b4588-af7d-4767-b710-f7b56f7abb27 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.160389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b4588-af7d-4767-b710-f7b56f7abb27 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.161806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 231/466] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:02.166943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b4588-af7d-4767-b710-f7b56f7abb27 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.199103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:02.270361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a1ffe-1db2-4b8b-bee0-da7cebfef4bd None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.296171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b4588-af7d-4767-b710-f7b56f7abb27 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.310398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.311307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.311766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.312207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.324610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.326530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.335753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.336672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.336672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.337127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.337795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:02.338108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.339643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-33c5a6fe-8a06-47e0-a8b9-d3930f05cd62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:02.344154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:02.345075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:02.345938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:02.345938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:02.345938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:02.347817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 236/467] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:02.349569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.350055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.350556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.358187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 232/468] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:02.358187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.374118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:02.375475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:02.428585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:02.462139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 237/469] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:02.466641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:02.467420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.475543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c864-ef55-4d10-847e-d085cef3851e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.510228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.511831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.512210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.512580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.513330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:02.513971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.516477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.531167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 233/470] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:02.532166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.592803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0def2bd-9b3c-43d7-a157-137bcc95baf0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.629495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.630296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.630670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.631033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.631784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.633311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.654301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99db4f-3312-4dbf-80f7-2a771df6127e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.666382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 238/471] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:28:02 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:02.667458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.724741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.725323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.725603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.725874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.737508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:02.751514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.753081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.769729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-11713717-05a1-49fa-8e3a-59e6c9451be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:02.776930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:02.777974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:02.778903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:02.778903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:02.778903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:02.780500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 234/472] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:02.782191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92106dab-9a16-40ec-a8eb-8d2fffe0dbc9 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.782476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92106dab-9a16-40ec-a8eb-8d2fffe0dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.782949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92106dab-9a16-40ec-a8eb-8d2fffe0dbc9 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.784630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 235/473] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:02.787346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92106dab-9a16-40ec-a8eb-8d2fffe0dbc9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:02.846720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae8094-2ddb-4bee-9396-30448d32280d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.897680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab2ae4-21ca-4f71-8d5c-356e0a9f3a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.898004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab2ae4-21ca-4f71-8d5c-356e0a9f3a2d None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.898361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab2ae4-21ca-4f71-8d5c-356e0a9f3a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.898579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab2ae4-21ca-4f71-8d5c-356e0a9f3a2d None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.899339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab2ae4-21ca-4f71-8d5c-356e0a9f3a2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.900937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab2ae4-21ca-4f71-8d5c-356e0a9f3a2d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.920978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92106dab-9a16-40ec-a8eb-8d2fffe0dbc9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:02.926233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 239/474] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:28:02 2022] POST /identity/v3/users => generated 248 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:02.927624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.928053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.928453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.947445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:02.949016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:02.964873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:02.966064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:02.966473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:02.966844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:02.967591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:02.967987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:02.969722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:02.988887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 236/475] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:02.994837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:03.041989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 240/476] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:03.075208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:03.106372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:03.117913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:03.182097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71936f-97d9-4239-b689-bbfb87811669 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:03.212371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10683a56-5bd2-4697-8e76-aec19d738320 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:03.224601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:03.225226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:03.225523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:03.225789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:03.226845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:03.228700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:03.255090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 237/477] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:28:02 2022] POST /identity/v3/users => generated 249 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:03.257135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:03.258007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:03.258364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:03.258715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:03.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:03.260874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:03.269941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.sql.core [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 02:28:03.269941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 02:28:03.269941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [parameters: {'id': '1c5312e30bfb4ad3b49347c5d8b984ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 02:28:03.269941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80496) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application conn.query(q) Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application result.read() Aug 09 02:28:03.286628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 02:28:03.287717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application next(self.gen) Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application yield resource Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application next(self.gen) Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise self.value Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application session.commit() Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 02:28:03.289251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self.session.flush() Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application compat.raise_( Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 02:28:03.290515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 02:28:03.291636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application conn.query(q) Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application result.read() Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 02:28:03.293069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [parameters: {'id': '1c5312e30bfb4ad3b49347c5d8b984ab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:03.294544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 02:28:03.296055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:03.297230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 241/478] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 9 02:28:03 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:03.298984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:03.421648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61625594-e1cd-47f8-ae61-e190293f752a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:03.440700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:03.469379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e957461-6537-4bd1-ad16-3042224282f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:03.470043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e957461-6537-4bd1-ad16-3042224282f0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:03.470375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e957461-6537-4bd1-ad16-3042224282f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:03.470648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e957461-6537-4bd1-ad16-3042224282f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:03.487818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e957461-6537-4bd1-ad16-3042224282f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-2e957461-6537-4bd1-ad16-3042224282f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:03.493198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:03.494291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 02:28:03.495156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:03.498246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 238/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:03.500204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:03.590614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:03.610405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eccd1-9472-424a-981a-f6644e0eb109 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:03.647843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:03.648690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:03.649065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:03.649499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:03.650313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:03.650716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:03.652245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189efd1-4ff5-46e2-99c4-94efa1b4a4e9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:03.661249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 242/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:04.649650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c9bc1-86ae-48e6-a803-9f2eb469fd12 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:04.650206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c9bc1-86ae-48e6-a803-9f2eb469fd12 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:04.650206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c9bc1-86ae-48e6-a803-9f2eb469fd12 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:04.651824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 239/481] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:04.659410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:04.659834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:04.660005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:04.681494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:04.682666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:04.762765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 243/482] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:04.792798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c9bc1-86ae-48e6-a803-9f2eb469fd12 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:04.898716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c9bc1-86ae-48e6-a803-9f2eb469fd12 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:04.933292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213981f6-a78a-4bc3-a075-fbf2cca0b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:04.934312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213981f6-a78a-4bc3-a075-fbf2cca0b07e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:04.934794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213981f6-a78a-4bc3-a075-fbf2cca0b07e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:04.935262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213981f6-a78a-4bc3-a075-fbf2cca0b07e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:04.946749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213981f6-a78a-4bc3-a075-fbf2cca0b07e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-213981f6-a78a-4bc3-a075-fbf2cca0b07e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:04.951831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:04.952753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:04.953578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:04.956333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 240/483] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:04.961903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.057387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5487-9ff8-40af-997c-0d38edb71b8d None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.058051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5487-9ff8-40af-997c-0d38edb71b8d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.058583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5487-9ff8-40af-997c-0d38edb71b8d None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.062253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 241/484] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:05.067450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.067982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.068455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.090545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:05.098920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:05.100167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:05.114268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884f162-165b-4491-93db-6ab4ae54c4c5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.160948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.163029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.163309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.163581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.164264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:05.164567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:05.166004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:05.170857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:05.188811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 244/485] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:05.194271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.244640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 242/486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:05.281377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.328537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff5880-7d84-49d9-80af-6887a0e9b139 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.379384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.380380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.380851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.381307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.383419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:05.394883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:05.396621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:05.423117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:05.424082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:05.424954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:05.424954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:05.424954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:05.427136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 245/487] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:05.433060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.497292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bcb7e-4ec2-4e2d-bcda-8522c6b3255a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.532628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e99e9-31cb-4f60-9bd3-3b0772a42a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.533556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e99e9-31cb-4f60-9bd3-3b0772a42a49 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.534100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e99e9-31cb-4f60-9bd3-3b0772a42a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.534404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e99e9-31cb-4f60-9bd3-3b0772a42a49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.547515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e99e9-31cb-4f60-9bd3-3b0772a42a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f27e99e9-31cb-4f60-9bd3-3b0772a42a49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:05.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:05.554592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:05.555417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:05.556068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfc4f8-bb82-4b4c-a99f-21cddd6c4537 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.556141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 243/488] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:05.556686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2cd91-9d3d-4aa0-97bc-d92b19c61c49 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.556778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2cd91-9d3d-4aa0-97bc-d92b19c61c49 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.557922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2cd91-9d3d-4aa0-97bc-d92b19c61c49 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.563504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 244/489] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:05.564313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2cd91-9d3d-4aa0-97bc-d92b19c61c49 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.593844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.594543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.594828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.595103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.595756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:05.596060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:05.597365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:05.597519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452165db-6ab4-4cf5-808a-8e582427b62a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:05.611193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 246/490] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:05.612578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.612870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.613149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.631874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:05.633139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:05.687366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:05.694981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2cd91-9d3d-4aa0-97bc-d92b19c61c49 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.718247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 247/491] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:05.719434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.754052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.754619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.754898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.755170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.755851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:05.756154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:05.757551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:05.766783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 245/492] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:05.767870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.859487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:05.871876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb69feb-7ccc-4dfe-ab01-fd8da5ab5e1a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.872434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd59cd3-b7c9-4411-915f-968d5d5f0770 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:05.906666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.907241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.907523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.907798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.916470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:05.917511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:05.918037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:05.918366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:05.918626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:05.923825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:05.924940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:05.925948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:05.930354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 246/493] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:05.930354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:05.935073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:05.936548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:05.952613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:05.953706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 248/494] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:05.955352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.004705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.044279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c624-6608-4517-9d68-cef1083836b2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.085787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.086921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.087448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.087971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.088464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a5a11-ae2a-4996-afcf-462f35cc8a8b None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.101401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.103221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.113047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:06.120153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:06.121094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:06.121968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:06.121968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:06.121968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:06.123536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 247/495] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:06.124637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.131124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.131884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.132216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.132443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.133184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:06.133492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.134936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37694fc-ba4b-4354-a362-124ce47dd932 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.143470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 249/496] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:06.148912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34d533-3cce-4feb-8008-4f82e822a714 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.149633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34d533-3cce-4feb-8008-4f82e822a714 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.149633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34d533-3cce-4feb-8008-4f82e822a714 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.154442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 250/497] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:06.154442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34d533-3cce-4feb-8008-4f82e822a714 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.232446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022a6c6-5259-45f3-be9d-e3b4a30b4c5e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.247293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.252162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34d533-3cce-4feb-8008-4f82e822a714 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.265253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.265800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.266069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.266363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.267184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:06.267489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.268943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcc6a7-05bc-4cd8-9561-a118f8020ff1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.284197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.284799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.285080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.285352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.285999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:06.286329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.287683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.298758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 251/498] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:06.299342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.306992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 248/499] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:06.308450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc7b4-d943-4772-a3a4-2427c5a4bffa None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.308784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc7b4-d943-4772-a3a4-2427c5a4bffa None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.309034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc7b4-d943-4772-a3a4-2427c5a4bffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.327126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c12bc7b4-d943-4772-a3a4-2427c5a4bffa None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:06.328346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c12bc7b4-d943-4772-a3a4-2427c5a4bffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:06.387149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.396887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6650-4d97-47ec-b9a1-82def708707f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.401196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 249/500] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:06.402552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f94ea-4323-4df8-a013-835a014ce2c0 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.402845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f94ea-4323-4df8-a013-835a014ce2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.403120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f94ea-4323-4df8-a013-835a014ce2c0 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.404601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 250/501] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:06.405108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f94ea-4323-4df8-a013-835a014ce2c0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.445207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.445850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.446202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.446509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.457319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.458856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:06.472974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:06.474271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:06.475127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:06.475127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:06.475127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:06.475127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 252/502] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:06.475352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.489013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.513969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f94ea-4323-4df8-a013-835a014ce2c0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.547310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.547870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.548171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.548441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.555351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.556847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-1632e329-ae07-423b-91e2-1bdc3a64b2a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:06.569218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:06.570139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:06.571159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 251/503] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:06.571747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.571925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.572173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.578609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf918e-a073-4e59-a069-bdf663662515 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.589305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:06.590518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:06.612957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.613560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.613839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.614107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.633004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.657090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 252/504] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:06.657902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.682393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.683938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.695100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.767060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fefe3a-2719-4986-9290-7f3e4097f986 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.780997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 253/505] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:06.781831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.800707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.801282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.801559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.801827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.812064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:06.816810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:06.817707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:06.818797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 253/506] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:06.819466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.862683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.881388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ba7ce-4061-4a12-a4ef-ccd722601dc4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.891885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:06.914693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.915330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.915609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.915877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.916526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:06.916825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.918182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.921909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421ee12-7605-4322-b5e9-fc1c1dd5fb8e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:06.929092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 254/507] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:06.929877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:06.956598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:06.957203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:06.957476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:06.957740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:06.958609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:06.958911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:06.960352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:06.977526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 254/508] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:06.978345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.030962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef98fe5-8c1a-451a-8b8c-c7ff31ef33ac None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.063711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.064277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.064564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.064839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.070739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.075384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.076824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.077056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54740-57f7-455c-873d-e077088292b9 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:07.091218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:07.092315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:07.093202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:07.093202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.093202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:07.093202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 255/509] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:07.093202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.115002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.115645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.115922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.116194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.118363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.126544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.127998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:07.143025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:07.144072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:07.145123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:07.145123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.145123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:07.145123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 255/510] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:07.145123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.196273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f75e-14d8-44cf-9736-b7d09efed720 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.230225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.230850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.231137 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.231414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.232094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:07.232396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.233788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.241519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 256/511] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:07.242377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.247017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b76f64-86f7-4ecd-8f57-0b84a8692921 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.281264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.281914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.282199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.282475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.289703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.291266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:07.301999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:07.303132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 02:28:07.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:07.307860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 256/512] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:07.308540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.320030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.326339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.342529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4e3e1-8f39-434d-8194-f06b744c1586 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.375252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.375862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.376141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.376414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.405753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d349-05b5-44cf-9d1c-59d2bf5a6911 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.439776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.440389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.440668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.440941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.441588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:07.441800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.441907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.443296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.443400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.460407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 257/513] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:07.461228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.526362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.531340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.542446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 257/514] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:07.543241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.567090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494967d-7e5f-4c46-b42a-d7f2fff22332 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.601390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.601986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.602296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.602595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.603243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:07.603542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.604920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.615753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 258/515] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:07.616520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.641211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda68bc1-19f9-4783-8512-2ebbb7292819 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.674429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.675057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.675334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.675606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.685596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.687084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:07.701667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:07.703087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:07.703948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:07.703948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.703948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:07.703948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 258/516] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:07.703948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.718478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f11c1-6fc7-4551-ae23-4e00fe1b097c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.738842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.744961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.753192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.753797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.754070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.754370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.755101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:07.755403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.756823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.773676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 259/517] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:07.774541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.804488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e8976-cb71-4034-bb5f-72aae61c1eae None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.836757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.837362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.837637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.837905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.838567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.840000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.861146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 259/518] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:28:07 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:07.861871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.884614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa263cb-1e6a-4e87-9d0e-9e1c2ceed040 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:07.926568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:07.927214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:07.927663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:07.927869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:07.940041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:07.941534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:07.958081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:07.959318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:07.960415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:07.960415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:07.960415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:07.960415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 260/519] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:07.960415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:07.972619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:07.986680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a6951-8164-4572-b3f2-c19eda8b78b7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.009796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.024026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.024218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.024446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.024724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.032010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.033668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:08.046813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:08.047871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:08.048948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 260/520] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:08.049508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.086586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a3507-393a-48d2-9cc6-ddcd92ab18aa None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.120609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.121477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.121758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.122032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.122717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:08.123131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.124418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.141279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 261/521] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:08.141279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.162764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5820fb-e65c-400d-a1d6-674527542481 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.197166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.197876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.198243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.198581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.220848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.235944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.240683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54768a14-720b-40d2-81b8-0c7bb0e7639e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.273000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.274704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.276204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.276916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.277936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.277936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.278345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:08.278721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.280140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.295688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 262/522] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:08.295688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.385258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 261/523] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:08.386361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.410297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed1909-7b69-4623-8c21-770518b3e934 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.444162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.445346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.445902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.446191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.446476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.447231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:08.447544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.448993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.471486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 263/524] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:08.471486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.487930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.500104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91399ebb-7f19-4301-abce-220434bc785e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.535490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.536161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.536515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.536787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.544191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.545767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:08.557977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:08.559209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 262/525] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:08.560141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.577184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1342b-5898-4f78-b4fb-a07934251d46 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.612873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.613515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.613790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.614058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.620761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.622235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:08.634775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:08.635741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 264/526] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:08.641768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.676178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94843d32-636c-46df-9a0e-3fe9902ceffe None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.690479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.713860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.714762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.714866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.715110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.727020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.730990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:08.732679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.754988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eecc28-694c-449e-be15-180f0da6a652 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:08.759037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:08.760132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:08.761193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:08.761193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:08.761193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:08.764857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 263/527] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:08.765587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.802255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.803479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.803929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.804389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.805284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=ad4a7ffff8f842969d782603cbb8fb89) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:08.805731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.807832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.822184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 265/528] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/role_assignments?user.id=ad4a7ffff8f842969d782603cbb8fb89&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:08.822184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:08.886758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e09b-647f-4458-8735-2c7fd562f72b None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.928000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:08.928733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:08.929009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:08.929289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:08.930025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:08.930545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:08.932235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:08.952906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d49f-e136-4609-ba46-f024002c04a7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:08.975307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 264/529] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:08.976273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.003996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.004781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.005151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.005566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.006397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:09.006702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.008164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.009272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.028755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 266/530] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:09.029643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.032730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.113559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc1457-743f-448b-9763-2a38b2da0389 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.140788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6ef60-e132-4c0a-a139-21d4805bf428 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.158623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.159248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.159932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.160211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.160861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:09.161161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.162571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.177614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.178412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.178791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.179234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.182586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 265/531] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:09.183480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.253946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.255611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.257626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.296069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.321624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eafeef-0c56-42f8-a269-764a424ec54f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.365209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 267/532] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:09.366137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.373556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.374353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.374735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.375111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.375974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=d57751a2455c48d28fcf9e97b62cfc38) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:09.376385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.377957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.384301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 266/533] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/role_assignments?user.id=d57751a2455c48d28fcf9e97b62cfc38&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:09.385280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.505792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75d9a4-7b7a-44fc-a4fc-2e4190c12f6d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.522942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90264447-0756-4713-98a0-72919d7f8f8e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.524771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.549479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.550616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.551854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.552494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.581819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.583969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.602696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.610131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.610944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.611128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:09.611207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:09.612033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:09.612905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:09.612905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:09.612905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:09.613068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.613143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 267/534] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:09.623082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.624458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.625921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:09.641873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:09.642834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:09.643711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:09.644629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 268/535] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:09.648643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.816117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12657e-0167-41ef-aabb-661c5bbdb0d0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.844249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e5d9-34fa-45a4-b212-9f6c40c20793 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:09.877176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.878060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.878533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.879473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.880601 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:09.881046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.882822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.895452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 268/536] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:09.901009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.914052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:09.915062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.915479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.915932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.916813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:09.917266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:09.918776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade769b-b266-42f6-af66-e659b5838e98 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:09.931810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 269/537] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:09.933178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c11b8-696d-4b9c-af74-99cead2e68d0 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:09.933481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c11b8-696d-4b9c-af74-99cead2e68d0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:09.933751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c11b8-696d-4b9c-af74-99cead2e68d0 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:09.938837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 270/538] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:09.938837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c11b8-696d-4b9c-af74-99cead2e68d0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:09.964817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:09.971093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:10.026347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03e65f-d634-4f5a-9d8a-16cfc0e0d937 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.052976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c11b8-696d-4b9c-af74-99cead2e68d0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.063043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.063790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.064069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.064342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.072558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:10.074057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-837e53e2-68c6-4f80-8e7f-bd1328fbd711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:10.087217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:10.088433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:10.089533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 269/539] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:10.090579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.090780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.090780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.090780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.091156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.093330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.093330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.093330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=7a0b055ed0d14d3e9428d1291f0f1209) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:10.093330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:10.095572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:10.106278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 271/540] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:09 2022] GET /identity/v3/role_assignments?user.id=7a0b055ed0d14d3e9428d1291f0f1209&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:10.108440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:10.115739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:10.117034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:10.213969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 270/541] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:10.257189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:10.262444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:10.300436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70b96d-e020-4a89-a6f0-5a59db76eeb8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.326669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:10.346948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.347569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.347846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.348115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.349191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:10.349495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:10.351027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:10.364314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 272/542] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:10.375236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:10.384079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52a547-b216-4cdf-9085-a322608a1758 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.431924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.432592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.432922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.433200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.451993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:10.464224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:10.466347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:10.468302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 271/543] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:10.475803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:10.585724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77cc40-ba46-48b1-bea1-54c91dd0bd66 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.622495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd68366-1bce-40be-a454-fbeafd20534d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.628340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.628959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.629238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.629511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.630283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=a425b3b4b3f34217ba148c2f855e0068) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:10.630594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:10.632254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:10.634389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:10.639734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 273/544] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:10 2022] GET /identity/v3/role_assignments?user.id=a425b3b4b3f34217ba148c2f855e0068&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:10.661212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.662033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.662442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.662814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.663628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:10.664036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:10.665644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:10.676100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 272/545] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:10.681793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:10.703936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:10.821360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e955-661f-41d6-9288-ed87b8ffc4c5 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:10.868873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:10.869500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:10.869731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:10.869962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:10.881965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:10.883469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:10.901968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:10.902936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:10.903819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:10.903819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:10.903819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:10.903819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 274/546] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:10.910217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.058442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47234b3d-b749-47c5-9c9f-dbaf26bcbd5f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.092258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.092877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.093159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.093435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.094084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:11.094415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:11.095791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb85b6-110f-490e-bdae-56b414808cca None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:11.115694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 273/547] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:11.126131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.222726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:11.239302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bcab4-3a40-466d-8f48-7448d5b5aa51 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.239599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bcab4-3a40-466d-8f48-7448d5b5aa51 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.239856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bcab4-3a40-466d-8f48-7448d5b5aa51 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.241936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 274/548] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:11.246779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.247067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.247340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.267472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc5441-daaa-4862-b03d-2dc71b9a551e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.282279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:11.283486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:11.301232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.301876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.302174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.302448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.310274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:11.311839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:11.322962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:11.323872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:11.325417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 275/549] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:11.332034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.347564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:11.427384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 275/550] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:11.448886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.554390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af7a5-3e74-4e8c-99c1-6b7dda38ec76 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.555801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b221dd-e6c2-435e-a572-8bf19a166a5f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.599697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:11.604250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.605010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.605289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.605561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.612812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.613580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.613861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.614138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.615237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:11.615551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:11.617022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:11.623311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:11.627990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:11.628913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:11.629926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 276/551] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:11.635015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.663116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 276/552] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 9 02:28:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:11.668909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.676528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:11.800793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352d96-70e5-428b-a57e-1158693da44a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.813848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22f958-c4bc-4edc-b496-94bd0d5dd7e2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.834934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.835554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.835832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.836102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.836813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:11.837164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:11.838558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89186-9392-4617-9551-e5de04c8dec6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:11.847184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 277/553] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:11.848874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399593f6-0bb2-439f-b497-5096be312b6f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.849158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399593f6-0bb2-439f-b497-5096be312b6f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.849430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399593f6-0bb2-439f-b497-5096be312b6f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.850904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 278/554] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:11.852836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399593f6-0bb2-439f-b497-5096be312b6f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:11.855000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.855634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.855916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.856190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:11.909420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:11.930459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:11.931920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:11.937409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:11.957169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399593f6-0bb2-439f-b497-5096be312b6f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:11.994681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:11.995617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:11.996316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:11.996724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.009830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.011739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-1b50ec0e-0375-4dea-b045-942856a8413f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:12.026810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:12.027958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:12.029039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:12.029039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:12.029039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:12.030777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 279/555] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:12.032193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.032611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.033028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.047166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 277/556] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:12.048158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.052167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:12.053537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:12.122871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 280/557] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:12.123960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.137156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.166950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d562c-c7e6-4f72-97da-fac3aee09406 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.194484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.201825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.202479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.202796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.203024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.203677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:12.203982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.205349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.217611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 278/558] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:12.222908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.236605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71e4d2-9de8-4b33-846d-37035c892b4c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.273911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.274858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.275275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.275679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.286422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.288036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:12.303025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:12.304519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:12.305409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:12.305409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:12.305409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:12.307637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 281/559] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:12.308677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.332709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceea6c2-5ca1-485b-a020-947c92d138b0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.367811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.368520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.368811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.369064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.376866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.382059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:12.387531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:12.388483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:12.389778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 279/560] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:12.390683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.432352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491abe3-ae10-45ee-b403-69ff316588a4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.449747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.468366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.469240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.469733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.470229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.479539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.481314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:12.493405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:12.494484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:12.495463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 282/561] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:12.496160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.498409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb449e-9c8b-411a-a8ae-62fb3a4f7e6c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.537188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.537976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.538497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.538623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.539286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:12.539596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.541120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26e3dc-1cda-4818-9e13-4fa489bfbed6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.553835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 280/562] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:12.554902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e25c44-2b9d-49fa-8349-f50745df8036 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.555193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e25c44-2b9d-49fa-8349-f50745df8036 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.555469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e25c44-2b9d-49fa-8349-f50745df8036 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.556825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 281/563] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:12.561347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e25c44-2b9d-49fa-8349-f50745df8036 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.614532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb94563-8f3e-4efa-a6f5-37b7e85a8a8d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.624678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.648860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.649571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.649848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.650116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.650806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:12.651126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.652479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.665384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 283/564] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:12.665384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.675574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e25c44-2b9d-49fa-8349-f50745df8036 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.710809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.711053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.711981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.712111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.712338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.713075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:12.713407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.714938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c71c5f-48e2-4548-bba6-e2f8f34b4f34 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.732519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 282/565] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:12.736189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.737049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.737482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.759581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:12.761041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:12.772998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01ab43-9747-458d-aa4c-25d3f6eaccf3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.810232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.811109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.811568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.812008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.820058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.822259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:12.842182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 283/566] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:12.842182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:12.842479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:12.843393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 284/567] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:12.844306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:12.903837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.948695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:12.952533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcac25-228f-447d-855c-960128ce24d7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.956231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab548c-3ccc-42af-9662-5a3de9c61c45 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:12.985480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.986061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.986379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.986649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.989482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:12.990113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:12.990424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:12.990696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:12.996674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:12.998114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.000439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.001863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:13.011971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.013093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:13.014018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:13.014018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:13.014018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:13.014018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 285/568] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.014540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:13.016844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.017823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:13.018776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:13.018776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:13.018776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:13.018776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 284/569] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.019175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.114989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.118377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7385113-113e-45a0-87ca-29eb74e5a9a3 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.120046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ae2bd-2290-4702-900d-f68ea35f93b1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.152338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8018ee3-dcde-414c-a602-f7fe0e822684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.152964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8018ee3-dcde-414c-a602-f7fe0e822684 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.153248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8018ee3-dcde-414c-a602-f7fe0e822684 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.153519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8018ee3-dcde-414c-a602-f7fe0e822684 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.154880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.155521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.155796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.156065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.156755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:13.157055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.158519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.163422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8018ee3-dcde-414c-a602-f7fe0e822684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e8018ee3-dcde-414c-a602-f7fe0e822684 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:13.168566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.169709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:13.170791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 285/570] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.171502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0b88-eba0-4452-8530-b53d9746bc1e None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.171579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0b88-eba0-4452-8530-b53d9746bc1e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.171843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0b88-eba0-4452-8530-b53d9746bc1e None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.173062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 286/571] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:13.173745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0b88-eba0-4452-8530-b53d9746bc1e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.183395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 286/572] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:13.183395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.191791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.277790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0b88-eba0-4452-8530-b53d9746bc1e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.286757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccb747-f366-4831-8afc-32469ef0ba07 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.310607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.311216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.311494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.311766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.312408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:13.312712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.314101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.321342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.321961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.322266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.322544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.323185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:13.323491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.324863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0fcc7-a3d5-4012-803c-89c268be732f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.325454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 287/573] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:13.326262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.340641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 287/574] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:13.340641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.340641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.340641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.349449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.355690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:13.356938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:13.417460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.422518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 288/575] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:13.424347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.431172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ca6f4-9f9e-4c13-a9b8-56c197e8f72f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.467961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.468600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.468878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.469156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.469806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:13.470104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.471539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.479514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 288/576] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:13.480283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.530508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1cd8d-879a-4d47-bc98-38f2fc1dfa31 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.564274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.567471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.568066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.568340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.568625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.569337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=5ea3e582aac94931b1a041817ccab685) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:13.569637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.571085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.576423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 289/577] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/role_assignments?user.id=5ea3e582aac94931b1a041817ccab685&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:13.577172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.582887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd1bbe-67d1-4a6a-86a1-c32f6a753821 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.622789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.623545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.623899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.624170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.632571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.634121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.644718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:13.648711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:13.651009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 289/578] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.651504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.698323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86137538-a9a0-4f89-9136-67efaff72338 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.737544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.738216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.738500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.738772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.747371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.748835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:13.760638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.761621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:13.762642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 290/579] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.763241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.772220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692620a-e661-49b2-a766-ca94602511f4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.810249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.810799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.811181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.811451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.822300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:13.827959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.828916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:13.829867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:13.834001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 290/580] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.834885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.837718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.868676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa8894-7581-4f33-95e9-4ef11d47c58d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.892771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:13.903375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.904023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.904304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.904580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.915431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.916899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:13.931759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:13.932766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:13.933685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:13.933685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:13.933685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:13.933685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 291/581] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:13.933685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:13.953470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b00c1-49d5-4149-8c43-599fec2511a8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:13.989013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:13.989653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:13.989910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:13.990422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:13.990881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:13.991187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:13.992731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.009269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 291/582] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.009269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.046513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cf0c9-59b2-4d1a-ac69-e52dc94660fb None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.071482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.081865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.082206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.082487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.082752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.083474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:14.083771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.085192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.102814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 292/583] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 9 02:28:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.103431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.116869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa044b-d887-4492-a94d-14b8d69e8c31 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.140738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.152215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.152829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.153107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.153378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.154051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:14.154384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.155757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.163561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 292/584] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.164309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.224914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335cb6-3f91-46b7-aac8-b70bdc902c09 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.263034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.263907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.264183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.264483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.265252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:14.265829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.267235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.279693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 293/585] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.280428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.283335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152dcd4-3b9f-4106-a3ff-6f3d6caf69b7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.309671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.321260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.321984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.322238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.322533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/ad4a7ffff8f842969d782603cbb8fb89/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.337502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=ad4a7ffff8f842969d782603cbb8fb89, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.338985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.362189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 293/586] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:14 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/ad4a7ffff8f842969d782603cbb8fb89/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:14.362189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.397724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.417822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80fa40-1391-4fa1-b891-7f1fb6ee308c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.458449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.459098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.459434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.459707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.472895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.474884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:14.490897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:14.491956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:14.492871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:14.492871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:14.492871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:14.497136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 294/587] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:14.498030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.507078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b721c00-f385-4007-a3a2-7c423a3bcb59 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.554653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.555295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.555637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.555953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.566506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.567966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:14.584667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:14.585692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:14.586641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:14.586641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:14.586641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:14.590267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 294/588] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:14.591145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.619756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.632461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb8a5e-6f1b-4279-a9ac-bec4cdfcf58f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.668661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.669338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.669624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.670005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.678928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.680567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:14.693788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:14.694924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 295/589] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:14.696472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.697132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c85ee-7aff-4294-a8eb-f5f80963f916 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.714505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.732484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.733107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.733389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.733671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.734438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:14.734858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.737089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.750897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 295/590] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.752018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.816283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f7af7-3304-4f8d-9aba-e0b26bccfde6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.837718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.852684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146b36b-0b4a-4380-98c9-0b285b919bb8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:14.853546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.854432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.855336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.856032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.856890 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:14.857412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.859046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.872567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 296/591] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.873748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.891984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:14.892606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:14.892888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:14.893167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:14.893982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:14.894322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:14.895799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:14.918001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 296/592] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:14.918001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:14.953060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:14.996024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752937f-08a2-4455-ab7b-cb1cb0037d5f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.038146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.038823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.039124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.039397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.048017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.049764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.054825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9890f9-25ef-4ef1-bf54-11617197e025 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:15.062515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:15.063419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:15.064345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:15.066953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 297/593] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:15.067954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.094695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.095258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.095544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.095816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.103885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.105227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:15.117155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:15.118274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 297/594] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:15.119430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.122802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.186145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9b739-9f19-4f73-8eeb-bb7b545001d7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.213879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.220104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792799ee-485f-4aa4-8716-aff67f543014 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.226061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.226912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.227341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.227852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.239904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.241660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.255388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.256010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.256293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.256568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.257271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:15.257577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:15.258190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:15.259442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:15.260569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:15.260569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:15.260569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:15.260759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.266516 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 298/595] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:15.267634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.272841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 298/596] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:15.273768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.335111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.373752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04f4c-f449-4fd5-99d5-1351b04e68ff None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.379863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f62a7-fd1e-44d7-958c-6a90233e31cc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.410928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.411558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.411841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.412132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.412771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:15.413074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.414494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.426020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.431150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.431289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 299/597] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:15.431289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.431952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.432440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.433357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:15.433826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.435449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.456844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 299/598] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:15.457932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.496743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.529702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687d187-2cd2-4e1c-8462-f214dfb13955 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.554451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.565510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.566208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.566498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.566775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/d57751a2455c48d28fcf9e97b62cfc38/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.576952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420211c6-e72f-4f0e-a4f8-349e9170f7d6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.581973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=d57751a2455c48d28fcf9e97b62cfc38, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.583468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.601738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 300/599] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:15 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/d57751a2455c48d28fcf9e97b62cfc38/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:15.602854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.617758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.618832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.619345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.619862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.628946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.630871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-01000883-5227-4cb6-a9ce-e8095c19cd10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:15.645595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:15.646840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:15.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:15.651296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 300/600] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:15.658673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e44f83-a6bd-4698-8eb5-bb8bdb19a271 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.659410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e44f83-a6bd-4698-8eb5-bb8bdb19a271 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.659938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e44f83-a6bd-4698-8eb5-bb8bdb19a271 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.662272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 301/601] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:15.663603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e44f83-a6bd-4698-8eb5-bb8bdb19a271 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.715507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6f2e-c15a-48ef-9f42-4d1faa241100 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.752578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.753232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.753513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.753789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.765097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.766875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.778711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e280e705-1240-48c3-8507-9144af5e6366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:15.783254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:15.784450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:15.785564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:15.785564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:15.785564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:15.785564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 301/602] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:15.785863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.785946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.786322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.790905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e44f83-a6bd-4698-8eb5-bb8bdb19a271 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:15.807957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:15.809177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:15.832457 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:15.836241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:15.837599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:15.838144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:15.838920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:15.839874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:15.840326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:15.841951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:15.855837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 302/603] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:15.857062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.887162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 302/604] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:15.887965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:15.978985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e325d-16b7-497d-804e-0914c17bbf89 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.007606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74199-608d-4130-8997-ecec45297e23 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.022915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.023544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.023829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.024356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.025034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:16.025340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.027232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.042997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 303/605] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:16.044019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.051013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.052057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.052543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.053053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/7a0b055ed0d14d3e9428d1291f0f1209/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.065367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.072494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=7a0b055ed0d14d3e9428d1291f0f1209, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.074888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.094527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 303/606] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:15 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/7a0b055ed0d14d3e9428d1291f0f1209/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:16.095485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.107901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.171134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3702ef-5162-41d5-893b-0fc0d77c94d9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.213759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.214453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.214737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.215015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.230678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446c9ad-dbd6-4ac4-9aa1-fa25e8672ead None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.235260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:16.246854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:16.247899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:16.248751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:16.252775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 304/607] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:16.262273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.321341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.322211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.322614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.322977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.339054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.340685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:16.360125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:16.361333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:16.362425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:16.366958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 304/608] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:16.372513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.373383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.397154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb0c0c-fb3d-4260-99f6-64eddc9ed622 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.443667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.444192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.444473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.444748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.445422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:16.445738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.447189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.465991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 305/609] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:16.472627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.511989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.549974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dad2e2-f886-44f1-856d-112e710f0368 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.592424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.593842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.594118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.594428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.595132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:16.595489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.596932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf6eb5-8173-41e7-bf42-03dc3fbf2690 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.615490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 305/610] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:16.616812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bab29-4024-4e66-94a7-f31a0db63773 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.617101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bab29-4024-4e66-94a7-f31a0db63773 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.617373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bab29-4024-4e66-94a7-f31a0db63773 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.619674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 306/611] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:16.623772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bab29-4024-4e66-94a7-f31a0db63773 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.630931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cc800-2545-40d5-b900-537e6d4117fa None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.666464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.667351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.667779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.668197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.675496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.680096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.682352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e8f50d49-03e3-439d-8845-6002b7036a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:16.697435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:16.698623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:16.699748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:16.699748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:16.699748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:16.701136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 306/612] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:16.702480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8237a-1357-409d-8750-782743837a79 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.702908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8237a-1357-409d-8750-782743837a79 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.703323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8237a-1357-409d-8750-782743837a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.721216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8ce8237a-1357-409d-8750-782743837a79 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:16.722719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8ce8237a-1357-409d-8750-782743837a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:16.749666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bab29-4024-4e66-94a7-f31a0db63773 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.777605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.788576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.789440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.789919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.790392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/a425b3b4b3f34217ba148c2f855e0068/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.813005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 307/613] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:16.813005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8237a-1357-409d-8750-782743837a79 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.813910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=a425b3b4b3f34217ba148c2f855e0068, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.815769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.844038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 307/614] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:16 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/a425b3b4b3f34217ba148c2f855e0068/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:16.845203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:16.921158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8237a-1357-409d-8750-782743837a79 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.946102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:16.960764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f22418-4066-43a0-a659-97cb1b54f7d9 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:16.962625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:16.963505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:16.963942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:16.964418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:16.965274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:16.965714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:16.967290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:16.980749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 308/615] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:16.989792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.004647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.004647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.004647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.004647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.014015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:17.019110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:17.020287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:17.021378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 308/616] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:17.026710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.086519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:17.156144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524c35a-f02c-4a59-9492-230dbc269c3d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.171559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275cf9b-9b76-45aa-8f90-e2f91e511c11 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.193047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.193667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.193942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.194232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.194908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:17.195206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:17.196623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:17.214904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 309/617] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:17.227220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.230356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.231145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.231433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.231707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.239649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:17.241110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:17.265585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:17.266603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:17.267637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:17.268182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 309/618] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:17.274075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.332035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:17.369735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:17.373916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd0c7f-091c-4b7f-9864-78bc58a6443c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.419899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.420868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.420868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.421190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.445970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:17.450891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:17.464829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2c6a3-de7a-4062-abf5-032ae0468268 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:17.478654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:17.479629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:17.480709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:17.480709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:17.480709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:17.490648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 310/619] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:17.495991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.506591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.507221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.507499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.507779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.508531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:17.508911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:17.514238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:17.537681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 310/620] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:17.543178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.654313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:17.660338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5c131-53c5-4fba-a012-26e8cf0c3bde None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.704283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.704931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.705288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.705470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.717487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307bbe9-9f5c-4043-b8d0-eb8ab4fd8ed1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.732233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:17.733630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:17.760947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:17.762002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:17.762959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:17.762959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:17.762959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:17.765996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:17.766886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:17.767392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:17.767885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:17.768754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:17.769286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:17.771298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:17.773955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 311/621] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:17.780197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.797659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 311/622] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:17.803085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:17.829150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:17.956360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e14ea-410c-4e56-9524-a938f88dbd55 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.982580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7ff7d-d2df-4a01-8d07-3a28265c4c22 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:17.999434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.000082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.000362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.000634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.012686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.014913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.023544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.024554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.024928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.025277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.025997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:18.026409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.027840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c89eb-c02a-4bbd-89ca-346c8bd7a49d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-0da50c0f-91e2-4cad-9dd6-2e5e1883f6d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:18.031227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:18.032187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:18.033176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:18.036241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 312/623] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:18.037664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b342-ba5f-4a86-a815-ee40af95b98b None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.038111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b342-ba5f-4a86-a815-ee40af95b98b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.038567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b342-ba5f-4a86-a815-ee40af95b98b None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.040364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 313/624] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:18.042463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b342-ba5f-4a86-a815-ee40af95b98b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.050530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 312/625] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:18.052016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.052376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.052724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.076230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:18.077844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:18.082419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.141196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.166175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 313/626] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:18.166175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.196741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b342-ba5f-4a86-a815-ee40af95b98b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.235872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.236509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.236796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.237066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.238080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:18.238241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.239655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.259057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 314/627] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:18.259913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.301198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548ec8d-039a-46b8-9ae4-30a06de59590 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.337159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.337798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.338076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.338389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.345166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.346842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.349663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:18.360614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:18.361684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 314/628] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:18.362657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.382543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27edb4-5c67-4a46-8573-e182d0466ced None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.410295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.419961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.420861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.421350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.421828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.432752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:18.437668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:18.438628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:18.439505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:18.447924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 315/629] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:18.447924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.468698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50fe7b-326e-4e1c-86d2-537bbc5c08e1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.503738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.504280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.504559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.504819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.558302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a035e-b866-4cd7-996a-b8555403356b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.570019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.573022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.574650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.595323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.595934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.596489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.596489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.597159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:18.597466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.599296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa26a8-5100-4a9e-94ba-b11f799cd3c6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.610897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 316/630] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:18.616012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb48f8-90c8-421c-bfe6-deff42d427cd None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.616299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb48f8-90c8-421c-bfe6-deff42d427cd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.616933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb48f8-90c8-421c-bfe6-deff42d427cd None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.622501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 317/631] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:18.622501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb48f8-90c8-421c-bfe6-deff42d427cd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.673275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.686357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 315/632] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:18.687258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.741113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb48f8-90c8-421c-bfe6-deff42d427cd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.776581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.777298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.777487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.777752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.778449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:18.778749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.780110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ac80-3a0c-40f1-a463-ab18cfc09581 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.788401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 318/633] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:18.789625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.789941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.790243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.797919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:18.800988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d640-857c-48fe-b78f-b3a583f4bc8d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:18.808101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:18.809397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:18.841099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:18.841800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:18.842083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:18.842390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/5ea3e582aac94931b1a041817ccab685/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:18.862981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=5ea3e582aac94931b1a041817ccab685, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:18.864838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:18.875965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 319/634] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:18.876885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.882357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 316/635] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:18 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/5ea3e582aac94931b1a041817ccab685/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:18.883412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:18.957281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.004626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccba69-8ba8-42c4-af4a-eca032d53577 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.037094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f7ca-3687-4648-9edb-014e2087e8b3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.045905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.047557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.048076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.048601 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.063493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.065685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.078718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.079394 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.079674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.079948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:19.084707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.085649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:19.086658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:19.086658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:19.086658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:19.088640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 317/636] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.089699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.091160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.092641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.093180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:19.120220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.121275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:19.122219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:19.122219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:19.122219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:19.122219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 320/637] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.122466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.209205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396559-c8fc-4fdb-b500-4ed2e08478d6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.219181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.229366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad98c9-d77d-4efc-b149-66c54652ff3c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.247475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.248105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.248379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.248642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.259097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:19.263807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.264721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:19.265574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:19.266837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.267303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.267580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.267848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.268490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:19.268789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.269822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 318/638] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.270283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa812b8-86e8-4abe-9119-a53cc4e5db12 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.270691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.282985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 321/639] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:19.284269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307a25-8ba0-4fff-a77d-81038f8a39a9 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.284556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307a25-8ba0-4fff-a77d-81038f8a39a9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.284826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307a25-8ba0-4fff-a77d-81038f8a39a9 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.286318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 322/640] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:19.286892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307a25-8ba0-4fff-a77d-81038f8a39a9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.339375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.382450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91697956-6eb3-4a8c-b0e6-42a9b479cd77 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.401510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307a25-8ba0-4fff-a77d-81038f8a39a9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.427419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.428060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.428308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.428811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.429235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:19.429538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.430977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.440704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.441306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.441583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.441853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.442717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 319/641] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:19.442826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:19.442904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.444390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd84b64-7d86-4791-9c5b-27cba696c2b1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.447522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.457501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 323/642] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:19.457501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.457501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.457501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.472726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:19.474324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:19.484743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.554187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 324/643] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:19.554187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.559123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf8eee-3028-4f45-8fa1-a19ddaf3ab18 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.584985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.596585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.597284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.597568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.597840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.606723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.608230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:19.620354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.621411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 320/644] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.622426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.664483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a756a17-2087-4d5b-8fa1-6217e8ad5add None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.702297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.703411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.703698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.703970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.712986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.714977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.721348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484f30c-3a22-4fe6-a21f-6afa8b7fa086 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:19.730325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.731404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 325/645] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.737467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.741076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.759332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.759952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.760227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.760492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.771403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.772983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:19.787267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.788364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:19.791180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:19.791180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:19.791180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:19.791180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 321/646] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.791180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.823919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:19.847406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981db78a-62f9-4ab4-9f23-7d90a31e2185 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.882776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.883413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.883789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.884067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.894355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49865714-96ce-4801-bcdb-b427e0234f94 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:19.895229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:19.899971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:19.900941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 326/647] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:19.906866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.934851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:19.935606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:19.935982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:19.936353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:19.937082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:19.937387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:19.938856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:19.956862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 322/648] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:19.957720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:19.985111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.028680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08353b-0f9c-4d96-9768-35092fe8e3ba None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.055032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.064101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.064627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da61c67-1de1-4b8f-ba26-d311521e4836 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.064747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.064986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.065258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.065951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:20.066294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.067718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.082372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 327/649] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:20.083199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.105432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.106359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.106871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.107355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.108268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:20.108786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.110400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.126593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 323/650] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:20.127544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.194120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0631bb7-ca4b-49bd-84b4-f8848db1d35b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.218123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.230451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.231079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.231442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.231601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.232237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:20.232538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.233891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.234333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1803-8d3b-458b-ac58-fd9f78a1365c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.246324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 328/651] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:20.246324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.273367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.273983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.274354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.274636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.297549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.346825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.348281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.350355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc47ab2-3e51-4589-a87b-1a0a87d6fd20 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.383801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.384625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.384898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.385167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.385901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=ad4a7ffff8f842969d782603cbb8fb89) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:20.386231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.387655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.393938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 329/652] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/role_assignments?user.id=ad4a7ffff8f842969d782603cbb8fb89&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:20.394865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.471801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.484965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 324/653] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:20.486428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.542086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf6f83-4468-4f8f-82a1-7873bb565c53 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.563188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.585957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.586638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.586965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.587238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.598995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.600445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.610347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807562bb-931f-42a2-b95e-1bb651ce8026 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:20.615199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:20.616627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 330/654] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:20.617667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.663068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.663609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.664735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.664735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.677804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.679815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:20.700786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:20.701805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:20.702777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:20.702777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:20.702777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:20.706219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 325/655] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:20.707249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.760160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c52e7-3bc0-4fea-83d7-d7cfadcb0ff8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.771986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.799240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.800420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.801043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.801518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.815447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.817426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.829838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836798b-247d-4fbe-bb0a-6c2bda76a494 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.832573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:20.835964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:20.837117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:20.838031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:20.838031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:20.838031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:20.838031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 331/656] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:20.838309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.876288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.877054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.877338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.877614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.878421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:20.878729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.880234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:20.899564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 326/657] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:20.900409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:20.948802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580d53d-e47b-4be5-b8b6-3cc07b2a4872 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:20.985745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:20.986758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:20.987276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:20.987761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:20.988623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:20.989001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:20.990478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39e012-bab8-4df3-aa48-d97cfa33f629 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.007712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 332/658] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:21.007914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e986ac2-b3fa-445e-823f-545ecc78f578 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.008002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4706b-6fad-4bfa-82ec-825fc0b10d63 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.008002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4706b-6fad-4bfa-82ec-825fc0b10d63 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.008002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4706b-6fad-4bfa-82ec-825fc0b10d63 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.008002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 333/659] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:21.008002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4706b-6fad-4bfa-82ec-825fc0b10d63 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.033850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.046094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.046817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.047231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.047505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.048282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:21.048589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.050070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.061496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 327/660] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:21.062335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.088314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.129467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4706b-6fad-4bfa-82ec-825fc0b10d63 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.167076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803930a-595a-4fd2-8ad2-d4b3e641c502 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.167423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.168026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.168303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.168579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.204009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.204588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.204865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.205137 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.214836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.216493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-9e2532f7-244c-449f-bb1a-656a07fb4f93 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:21.228879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:21.229862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:21.230879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 328/661] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:21.231496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.231496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.232151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.242934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.244488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.250217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:21.251475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:21.268261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.326200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 329/662] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:21.326200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.338365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.373903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 334/663] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:21.373903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.435155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f5f92-67bd-4ccc-abae-be7b41fe1b18 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.472307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.472857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.473133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.473398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.480573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.480706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc9dc5-e8da-4041-af36-476ac667cdce None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.482013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:21.493309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:21.494523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 330/664] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:21.495491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.509482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.522207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.522818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.523091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.523359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.524111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:21.524428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.525874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.544606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 335/665] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:21.545439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.576384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.607746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15745bef-c9ba-4540-986d-d87071bec207 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.643636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.644307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.644577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.644845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.654046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a10922-73bb-4818-8592-f208c3f04c8a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.655145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:21.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:21.661226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:21.662114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 331/666] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:21.662863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.692095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.692738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.693026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.693301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.705367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.706855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:21.721986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:21.723046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:21.724311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:21.724311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:21.724311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:21.724311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 336/667] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:21.724311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.758321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.776480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d08ea-7e4c-4b82-bfdd-6ab24bd8c946 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.805183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.811031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.811600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.811878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.812376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.812832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:21.813131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.814613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.831736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 332/668] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:21.831736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:21.835346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e1db7-4c3a-4c4a-944e-762f22862f57 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.869130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.869735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.870011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.870314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.933277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c19b-db33-489e-99e1-22693e573cc9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:21.940462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.942393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.967367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:21.968083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:21.968362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:21.968630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:21.969395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:21.969695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:21.971189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:21.972178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:21.983483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 333/669] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:21.983483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.042178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.056319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 337/670] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:22.057065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.088336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742dcb-169e-47da-91d1-fb20d5ba69f3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.123757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.124365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.124638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.124886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.125542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:22.125839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.127305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.141298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 334/671] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:22.142123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.168253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b8fff-a1ec-42ac-b012-1feea53d39f7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.204997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.211061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.211676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.211981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.212251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.212981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=d57751a2455c48d28fcf9e97b62cfc38) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:22.213283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.214752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.220524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 338/672] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/role_assignments?user.id=d57751a2455c48d28fcf9e97b62cfc38&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:22.221298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.257026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a22de-4dbe-4d92-adda-ddf648e6169b None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.274219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.305503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.306373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.306653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.306924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.319218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.320697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:22.338582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:22.339807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:22.340738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:22.340738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:22.340738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:22.342777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 335/673] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:22.343889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.362553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865c358-151e-43e4-9de6-5f048b92eb8e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.399065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.400159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.400630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.401099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.421954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.424673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:22.442727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:22.443742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:22.444670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:22.444670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:22.444670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:22.451825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 339/674] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:22.451825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.467884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b52dbe-19f3-434d-88bb-69963b3a9ee2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.508596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.512654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.513584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.514085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.514933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.523086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.524687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:22.536355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:22.537293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:22.538279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 336/675] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:22.538797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.565101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624de92-32d1-428b-9456-1c11b7d652ed None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.567184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.607315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.607936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.608214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.608490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.609139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:22.609444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.610856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6ecff-43ea-4f42-9e16-85ca9172f077 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.627737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 340/676] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:22.627737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87176ca-c490-49ba-a0a4-01ab2c1e0e87 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.627737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87176ca-c490-49ba-a0a4-01ab2c1e0e87 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.627737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87176ca-c490-49ba-a0a4-01ab2c1e0e87 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.630417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 341/677] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:22.631205 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87176ca-c490-49ba-a0a4-01ab2c1e0e87 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.645488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652ded-e768-4c2d-8dc7-2330f63b220d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.681706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.682690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.683174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.683645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.684465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:22.684943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.686522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.698913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 337/678] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:22.700039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.737917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87176ca-c490-49ba-a0a4-01ab2c1e0e87 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.747472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.773984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.775401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.775401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.775401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.776132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:22.776504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.778003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d1297-5495-44f9-a19e-f8ba035f3401 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.789753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 342/679] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:22.792732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:22.795821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.796114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.796392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.809818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef662cc-08b0-4881-8461-e78605e34aff None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:22.813844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:22.815071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:22.846738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:22.847394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:22.847827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:22.848110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:22.856164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:22.857777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:22.871983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:22.873118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 338/680] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:22.874083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:22.894194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 343/681] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:22.894194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.002423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.007168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f90dd-f64b-45d7-9528-75b1cb84a37e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.046117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.046681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.046965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.047236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.047973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=7a0b055ed0d14d3e9428d1291f0f1209) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:23.048284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.050335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.066505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 344/682] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/role_assignments?user.id=7a0b055ed0d14d3e9428d1291f0f1209&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:23.066505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.122645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb891d7-a220-44ed-974e-7d4405e180f1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.161147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.161954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.162391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.162792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.170681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.172399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:23.184714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:23.185920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:23.186885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:23.187342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 339/683] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:23.188435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.193527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf6df8-a860-4884-9d11-06c28be52cb4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.198007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.231899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.232854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.233376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.233861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.234906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:23.235437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.237184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.257596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 345/684] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:23.258947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.282513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.300731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655cfcc-d34e-41dc-b3d1-0cc9c446cefe None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.340213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.341793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.342554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.342554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.356672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:23.361406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:23.362393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:23.363276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 340/685] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:23.363996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.386895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb4053-93df-41a3-a753-7be693405841 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.433889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.434559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.434837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.435103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.435821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:23.436144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.437613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.450015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.451054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 346/686] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:23.451824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.501111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf38c3b-a99f-4ce9-b0e1-76912ecc0719 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.537449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.538247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.538360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.538633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.566828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.594044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f977432-0bdd-4a82-ad37-e75bfbc58a9f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.612119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.613817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.639743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.640946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.641821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.642580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.643710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:23.644611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.646556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.658405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 347/687] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:23.660260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.679578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.726431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 341/688] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:23.727492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.778812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b99ff5-df4d-4629-a75a-d1c577cef5af None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.813848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.814508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.814856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.815056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.815788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=a425b3b4b3f34217ba148c2f855e0068) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:23.816099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.817579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.830582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 348/689] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/role_assignments?user.id=a425b3b4b3f34217ba148c2f855e0068&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:23.831562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.849844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:23.853694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2f742-829e-4f05-9d08-6719cf8f7501 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:23.898008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:23.899592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:23.900102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:23.900714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:23.913797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:23.915695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:23.934724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:23.935782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:23.936703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:23.936703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:23.936703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:23.939221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 342/690] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:23.948011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:23.996921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.066718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8929e2-7fa7-4414-8ec1-9e69d8943b33 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.106410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214bfa9-5442-4d10-a170-4acc3df32d51 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.114426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.115188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.115571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.117013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.145753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.147245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:24.166291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:24.167376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:24.168332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:24.168332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:24.168332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:24.168332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 349/691] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:24.169683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.170663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.171313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.171776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.172620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:24.173133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.173236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.176143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.191756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 343/692] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:24.197032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.249950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.292673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.301445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc9946-34ba-4f57-99cd-07938398ca5f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.323845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be13d9-196a-4ea9-be73-10aa0d58ec51 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.345752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.346757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.349358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.349847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.350749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:24.351271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.355458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c633d-d2ac-44fb-815a-5ed8cf4f40f0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.376061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 350/693] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:24.377677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28c60-f3fb-435e-9971-4bc3717cac6b None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.378301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28c60-f3fb-435e-9971-4bc3717cac6b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.378779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28c60-f3fb-435e-9971-4bc3717cac6b None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.380655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 351/694] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:24.381619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28c60-f3fb-435e-9971-4bc3717cac6b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.382929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.384294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.386687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.386918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.396346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.398035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-8bfa37c9-f19f-4e95-919f-c2e3a1a30790 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:24.409650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:24.410721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:24.411692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 344/695] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:24.412308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.413043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.413043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.441986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:24.443232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:24.510028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28c60-f3fb-435e-9971-4bc3717cac6b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.529168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.534371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 345/696] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:24.535356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.566667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.567632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.567917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.568191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.577285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.579033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:24.592683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:24.593613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:24.594583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 352/697] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:24.595079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.624621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.666580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62dd5e-61a9-4a61-9b8e-b2a320a1d94c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.702553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.703487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.703609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.703896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.704634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:24.704944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.706500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.710089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76c553-eeeb-46a5-a84a-9b1a326f7ae5 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.730420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 346/698] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:24.731715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.746616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.747627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.748052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.748553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.749287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.750915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:24.763182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:24.764435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:24.765480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:24.767744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 353/699] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:24.768739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.813425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.845557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82714fca-5c2c-4fab-b25c-7e4ea2e7c478 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.873079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcc867-1540-4984-8a18-29669794b49b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:24.874184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:24.884357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.884978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.885261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.885533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.886442 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:24.886662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.888148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:24.905843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 347/700] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:24.905843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:24.916485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:24.917368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:24.917795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:24.918309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:24.988761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:24.990495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.009911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0b2e2-0b62-4536-8577-4c8c024fdc59 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:25.048847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:25.049494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.049778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.050052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.050741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:25.051049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:25.052459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.062817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 348/701] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:25.063900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:25.120695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:25.158272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:25.179119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d1275-1d55-4e50-9e16-9869eb6d1a14 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:25.216350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:25.216822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.217448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.217727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.220301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=5ea3e582aac94931b1a041817ccab685) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:25.220301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:25.220806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.223179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 354/702] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:25.224760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:25.242250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 349/703] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/role_assignments?user.id=5ea3e582aac94931b1a041817ccab685&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:25.242250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:25.391581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dce847-ed08-4ceb-9e6e-c3c7d4765068 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:25.395237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab1087-bf14-43a7-a089-aec10fb09f1a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:25.419481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:25.431104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:25.432633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.432633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.432633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.432898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:25.433109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:25.434761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39040-cb3e-4501-baf2-aff7b34bb452 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.445582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 355/704] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:25.447346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0d65e-7c2d-4e0c-93ef-5889871edba1 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.447615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0d65e-7c2d-4e0c-93ef-5889871edba1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.450202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0d65e-7c2d-4e0c-93ef-5889871edba1 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.455055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:25.455055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.455055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.455410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.458358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 356/705] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:25.460166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.460675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.461152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.482329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:25.483420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:25.485123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.499864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:25.507547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:25.508652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:25.509752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:25.509752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:25.509752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:25.510964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:25.516336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 350/706] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:25.522616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:25.637202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 357/707] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:25.678945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:25.723154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502380cb-1a98-48c6-8e1c-1af6753755d8 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:25.768307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:25.769963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.771184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.771698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.772808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:25.773919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:25.775851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.792894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 351/708] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:25.800169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:25.819396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99290665-8052-4aaa-8656-50e1b4498c28 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:25.862928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:25.881077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:25.881743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:25.882018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:25.882902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:25.900631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:25.902115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:25.902858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:25.931000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:25.932120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:25.933069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:25.933069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:25.933069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:25.940509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 358/709] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:25.941442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:26.005113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a11212-07d6-4155-9926-ab8eeea55036 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.078756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.079400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.079675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.079942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.097770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.099628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:26.113239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:26.114316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:26.115258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 352/710] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:26.121935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:26.145206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e297ed-fc09-4b92-accf-bcfb2e0952ff None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.200731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.201351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.201625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.201896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.202583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:26.202883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.204254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac377-2c7a-4840-acde-0b19d872447e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.218446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 359/711] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:26.222188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adecac4-c5f2-4c42-be54-900988773bce None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.222502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adecac4-c5f2-4c42-be54-900988773bce None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.222776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adecac4-c5f2-4c42-be54-900988773bce None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.226247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 360/712] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:26.227046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adecac4-c5f2-4c42-be54-900988773bce None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:26.243866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:26.262498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1a577-7a90-41b2-9580-04df4d42e4a5 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.285116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:26.301022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.301958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.302582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.303236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.304188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:26.304710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.306530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4969430-4d4a-4a32-b7d0-72b24d7cd879 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.325107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 353/713] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:26.326923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.327513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.328022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.339051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adecac4-c5f2-4c42-be54-900988773bce None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.349768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:26.351409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:26.374330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.374952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.375229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.375503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.376153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.377591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.406480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 361/714] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:28:26 2022] POST /identity/v3/users => generated 254 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:26.421270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:26.434774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 354/715] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:26.468673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:26.614998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:26.619481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06649bc2-470e-4185-9b19-8a8617368a73 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.623905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:26.647316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630091f-22c3-437c-a72b-10ffc8ee53b6 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.670541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.671533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.672142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.673176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.674426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=ad4a7ffff8f842969d782603cbb8fb89) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:26.674973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.677534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.686535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 362/716] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:26 2022] GET /identity/v3/role_assignments?user.id=ad4a7ffff8f842969d782603cbb8fb89&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:26.691956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.692625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.692907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.693180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.693783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.695215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:26.706898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:26.708034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:26.708923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 355/717] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:26 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:26.715195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:26.849845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1d80-5f1a-4b20-a650-28309aeabb69 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:26.888480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:26.889029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:26.889303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:26.889570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:26.890249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] RBAC: Adding query filter params (name=network) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:26.890606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:26.892032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710f16e-e0f6-4066-ab8d-ec9235f83b3a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:26.901957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 363/718] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:26.903356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:27.042111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443652-fa60-41c6-835d-f2921e08c6f4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:27.091153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:27.092157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:27.092638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:27.093113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:27.093961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] RBAC: Adding query filter params (type=network) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:27.094475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:27.096035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c92f32-1da5-484a-b5cc-516b225fdd87 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:27.106936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 356/719] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:27.871735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a80f5-587e-4ea7-ad98-c7770a6c56ac None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:27.872299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a80f5-587e-4ea7-ad98-c7770a6c56ac None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:27.872299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a80f5-587e-4ea7-ad98-c7770a6c56ac None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:27.873924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 364/720] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:27.876608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52cb8b-c85a-4e62-bfa1-ac5f84938bda None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:27.876896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52cb8b-c85a-4e62-bfa1-ac5f84938bda None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:27.877169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52cb8b-c85a-4e62-bfa1-ac5f84938bda None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:27.879540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:27.879817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:27.880086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:27.888769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 357/721] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:27.900043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:27.900043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:27.900043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:27.901791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:27.903007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:27.910546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:27.929872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:27.931754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:27.940276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:28.000496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 365/722] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:28.026894 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:28.046024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 358/723] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 02:28:28.067855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:28.168103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed330fe-c46d-42a9-a283-67b47747c7cd None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:28.202629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:28.203639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:28.204122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:28.204597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:28.205451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:28.207045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:28.210671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:28.218172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b6af7-340d-4a89-a0c5-403cf516805d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:28.236477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 359/724] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:28:28 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:28:28.269573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:28.270353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:28.270633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:28.270907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:28.283699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:28.288738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:28.289725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:28.290719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 366/725] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:28.298059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:28.413218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8bea-e45d-45ac-a956-5679649c5b16 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:28.459262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:28.459896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:28.460174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:28.460514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:28.461199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:28.463529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:28.463529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:28.473230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 360/726] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:28.478920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:28.511325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:28.583363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05449ad-47cb-4dcd-ba15-894286f83c74 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:28.617936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:28.618684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:28.619223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:28.619501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:28.630246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:28.631712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:28.646640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:28.647632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:28.648572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:28.648572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:28.648572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:28.648572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 367/727] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:28.655488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:28.717006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:28.763215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff87908-55d2-4c68-9042-c38b7fe679cb None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:28.798321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:28.798938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:28.799218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:28.799492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:28.800145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:28.800446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:28.801813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:28.814631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 361/728] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:28.823502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:28.898209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:28.930483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2243455-5e91-47fd-bbb8-2c8d95b06df4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:28.965105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:28.965739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:28.966018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:28.966319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:28.974614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:28.975991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ae552316-1b6e-493c-a923-8675488aaf02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:28.987343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:28.988342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:28.989311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 368/729] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:28.994440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.046434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773df16d-d2e4-4eab-aeeb-8e34071f0d15 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.046777 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773df16d-d2e4-4eab-aeeb-8e34071f0d15 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.046987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773df16d-d2e4-4eab-aeeb-8e34071f0d15 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.048406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 369/730] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:29.053332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7215d9-b71b-4d19-993d-4a007397852d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.053611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7215d9-b71b-4d19-993d-4a007397852d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.053881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7215d9-b71b-4d19-993d-4a007397852d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.085526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4f7215d9-b71b-4d19-993d-4a007397852d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:29.086811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4f7215d9-b71b-4d19-993d-4a007397852d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:29.160317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6946d-f99e-4756-bb1e-226205383e0d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.174008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 370/731] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:29.175413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64899172-8bfb-452e-904e-d45f1f67f39f None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.177810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64899172-8bfb-452e-904e-d45f1f67f39f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.177810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64899172-8bfb-452e-904e-d45f1f67f39f None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.178103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 371/732] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:29.190508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.190783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.191059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.201114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:29.206439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.207069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.207352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.207616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.208346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:29.208646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:29.210126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:29.219649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:29.221104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:29.228653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 362/733] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 9 02:28:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:29.229573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.287765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:29.306262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 372/734] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:29.307180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.347489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34148c-9705-44c8-ada0-0ec4d2b9bfec None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.385496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.386536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.387034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.387560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.399006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:29.415455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:29.416394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:29.417266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:29.420114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 363/735] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:29.425095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80ea7b-8817-4964-9e6b-cff93fc35436 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.426420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.455765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.456405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.456686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.456952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.466845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:29.521494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:29.532141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:29.533615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:29.559153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e49cf-a42d-4017-9d03-9098b4df39fe None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.605439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.606182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.606393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.606664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.617041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:29.621760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:29.622861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:29.624079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:29.627798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 364/736] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:29.628729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.658652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 373/737] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 02:28:29.663571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.751440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:29.753615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:29.762274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25737-5d5d-4638-9b0f-cafcd688c235 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.762508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4b7b-4ea9-45f4-9c75-a901971a96f0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.798808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.799349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.799608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.799885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.800722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:29.801026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:29.801999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.802428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:29.803098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.803598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.804073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.804932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:29.805443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:29.807064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:29.810688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 374/738] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:29.811504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.816747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 365/739] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:29.822729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.921430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f3fcf-7e12-4ec3-9e55-48e646696872 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.935201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb3647-56d9-4782-bf4c-61a4e86d7e88 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:29.958100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.959074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.959362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.959633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.969798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:29.970419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:29.970551 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:29.970743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:29.971000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:29.971880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:29.982015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:29.983472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:29.986713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:29.987649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:29.988585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:29.988585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:29.988585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:29.988585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 375/740] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:29.989294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:29.993282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e00fa85c-2907-4289-b585-0a64180f8e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:30.002721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:30.004111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:30.005033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:30.005033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:30.005033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:30.005033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 366/741] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:30.005033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62bc55-4355-4857-a675-94fc672da39e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.005336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62bc55-4355-4857-a675-94fc672da39e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.005405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62bc55-4355-4857-a675-94fc672da39e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.006703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 367/742] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:30.007281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62bc55-4355-4857-a675-94fc672da39e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.018421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.104107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a0fad-ea6b-4aea-a7fc-3a71b77ee584 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.116468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62bc55-4355-4857-a675-94fc672da39e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.140111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.140753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.141032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.141366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.151718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.152453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.152588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.152791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.153022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.153692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:30.154090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.154240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.155529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.172446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 368/743] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:30.172446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-6d1e4ed2-51a8-49da-ba9a-622c9b4ff770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:30.172689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:30.173605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 376/744] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.174546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.199167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:30.200720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:30.248539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.256373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.273022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 377/745] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 02:28:30.273859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.299915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fbc9c-2fa5-47ad-9283-25fcf5094be7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.335511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.336524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.337076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.337637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.338504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:30.339014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.340789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.354103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 369/746] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:30.355383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.381819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25fd6-9c1e-48d0-a7ef-5e5397cf8101 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.425908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.426594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.426871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.427156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.433935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.435410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:30.447565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:30.448489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 378/747] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:30.449421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.473342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda5c87-8dba-49cf-975c-ab9f298f5096 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.500710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.501968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.511914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.512633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.512916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.513189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.513843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:30.514145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.515540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.527813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 370/748] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:30.528629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.569717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cea437-df02-40b1-92f9-44ddc74b8d6c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.607892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.608424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.608702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.608975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.609621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.611027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.627031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 379/749] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:28:30 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:30.627890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.647927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00bb2b-e4e4-49cd-8fce-2f12454b4e60 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.702624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.703344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.703627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.703902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.713174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.714733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:30.727837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:30.728883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:30.729885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 371/750] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:30.730561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.761688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165d571-a431-4258-aff6-5cf7c97d232a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.770454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.781717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:30.806653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.806871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.807244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.807523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.808231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:30.808535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.809971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f39f-3710-43d0-afad-364927f16ef4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.822782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 380/751] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:30.824151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b6a3-15aa-4c74-a08c-80e50360a092 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.824441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b6a3-15aa-4c74-a08c-80e50360a092 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.824709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b6a3-15aa-4c74-a08c-80e50360a092 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.827895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 381/752] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:30.828652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b6a3-15aa-4c74-a08c-80e50360a092 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.846963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985b344-923c-4d54-b4c0-5d28ba13031e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.887334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.887892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.888172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.888446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.896478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.897915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:30.909084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:30.910072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 372/753] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:30.911104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:30.938421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b6a3-15aa-4c74-a08c-80e50360a092 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:30.975526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:30.976226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.976606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.976885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:30.977675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:30.977987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:30.979439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d0262-16e2-4a77-8913-44ac4a1da23f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:30.996795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 382/754] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:30.998085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:30.998401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:30.998672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.017202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:31.018494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:31.020602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b487c8-a4f3-4eb6-9b5f-8814000e00d2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.023659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.037505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.058484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.059098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.059371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.059636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.060362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=5ea3e582aac94931b1a041817ccab685) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:31.060665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.062086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.078200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 373/755] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:30 2022] GET /identity/v3/role_assignments?user.id=5ea3e582aac94931b1a041817ccab685&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:31.078200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.107200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 383/756] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:31.108216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.223406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bc6d4-7b28-437b-831f-1fcb37871160 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.224859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc67af-ceaa-4e5c-98e2-44b5c7318f6f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.265066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.265775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.266122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.266488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.266690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.267319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.267431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:31.267710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.267817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.268096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.269365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.294258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 374/757] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:31.295170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.306360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.325335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.357203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.359192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.399065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dd533-14dc-42a9-9df7-e07095d37b3e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.436974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.437677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.437964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.438274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.438870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.440239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:31.449911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:31.450953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:31.451820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:31.456030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 375/758] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:31.456878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.472453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 384/759] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:28:31.474145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.532726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.561230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.575551 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569be928-4fdc-4315-b5ac-3352649c1040 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.583218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610c66a-18b7-452e-8350-fb79a8e60307 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.611301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.612253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.612742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.613265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.623240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.624554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.624813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.625552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.626262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.626384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.627217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:31.627910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.630008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.640642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 385/760] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:31.641564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:31.642323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:31.643255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:31.644166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:31.644166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:31.644166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:31.650706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 376/761] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:31.651836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.752693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb9074-7d8d-4d09-b859-6244621aadc8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.770706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.771926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864a1a5-bacc-4df5-827e-2cbc5d9c02c3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.790486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.791207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.791493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.791821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.803040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.804520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.813682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.814656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.815143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.815610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.816438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:31.816935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.818538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:31.819666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:31.820596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:31.822240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:31.822240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:31.822240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:31.822240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 386/762] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:31.822240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.827702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 377/763] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:31.828769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.838504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:31.929426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e2603-8331-4c11-82a5-c57f1d596eb1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.935515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348908d-9aac-4820-b5a4-fd91234fa2a9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:31.962979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.963579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.963861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.964134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.964910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:31.965082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.966642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.973303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:31.974565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:31.975054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:31.975525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:31.976408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:31.976899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:31.978583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:31.983582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 387/764] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:31.983582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:31.996932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 378/765] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:31.998182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.010047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:32.068584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:32.084028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea8701-0289-44e5-b325-e960780283c7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.106103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b5932-ff6d-4b23-9dcc-2694c0be2951 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.122038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.122700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.122977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.123253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.123952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] RBAC: Adding query filter params (interface=public, service_id=be416c1e53b046b2b9684912bb411b9d, region_id=RegionOne) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:32.124257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.125614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.138350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 388/766] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/endpoints?service_id=be416c1e53b046b2b9684912bb411b9d&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:32.139254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.144426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.145284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.145785 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.146263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.153879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.155825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:32.168842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:32.169770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:32.170715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:32.177550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 379/767] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:32.178600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.238843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:32.245895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448b2e-84a7-4a4f-90ec-ea6c6f663370 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.280223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.280883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.281228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.281569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.289307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.292010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.303641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42563f02-1874-460a-ae99-ec3d47012fde None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:32.305305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:32.306362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:32.307486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 389/768] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:32.308372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.334620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:32.344502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.345379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.345867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.346375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.347295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.349233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.355517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 380/769] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:32 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:32.356620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.477788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6974d-b341-4651-821e-a97b93c1792f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.516378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.517430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.517926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.518638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.519536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:32.520058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.521701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.536411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151928c2-e072-4859-bbda-852ea179f014 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.536807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 381/770] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:32.545203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.608592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.609245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.609735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.609843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.610511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:32.610815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.612244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.630179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 390/771] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:32.630375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:32.632767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:32.650552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:32.694888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5a71-2ba4-4b5e-ab98-e5e8aa71b2b4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.746471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.747096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.747371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.747637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.748473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=562c6203e9cf440ebd81d14877c9e452) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:32.748667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.750090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b1911-8f55-4479-a927-8a1b8e455ed6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.757814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32126a-4f7b-4cd0-854c-9e4937723cc2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:32.759419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 382/772] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:32 2022] GET /identity/v3/role_assignments?user.id=562c6203e9cf440ebd81d14877c9e452&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:32.761841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d096d1e-abfb-4f25-8775-b6bd6bdf132f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.762132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d096d1e-abfb-4f25-8775-b6bd6bdf132f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.762452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d096d1e-abfb-4f25-8775-b6bd6bdf132f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.764193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 383/773] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:32.769664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.770001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.770247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.796127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:32.797528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:32.806120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:32.807229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:32.807722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:32.808192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/ad4a7ffff8f842969d782603cbb8fb89/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:32.810550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=ad4a7ffff8f842969d782603cbb8fb89, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:32.812339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:32.842316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 391/774] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:32 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/ad4a7ffff8f842969d782603cbb8fb89/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:32.884970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 384/775] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:32.912306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.012417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:33.048148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac297f93-3288-4b1d-9744-3f31a5d43d0e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:33.103976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:33.104988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.105461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.105927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] PATH_INFO: `/v3/services/network` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.106743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:33.108359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e3dc7439-8c64-437b-8e85-f897b67c2558 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:33.118307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:33.119327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:33.120180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:33.122999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 392/776] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:32 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:33.129021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.197749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:33.221881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7a633-0e5e-49c7-acf4-3accd9484b47 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.222245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7a633-0e5e-49c7-acf4-3accd9484b47 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.222450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7a633-0e5e-49c7-acf4-3accd9484b47 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.225363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 393/777] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:33.230349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.230759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.231143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.266119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:33.267537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:33.273476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950cf99-64cf-4d62-972e-69f4a85e3d81 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:33.321329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:33.322216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.322644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.323302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.323954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] RBAC: Adding query filter params (name=network) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:33.324275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:33.325800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:33.339477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 385/778] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:33.346449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.348733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 394/779] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:33.395577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.503723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:33.506764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee10f-b895-45fd-966d-1587ea50b33a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:33.522489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:33.560792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aa4e7-0ff9-4b82-88ad-de07463cb3b0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:33.562794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:33.563509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.563784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.564054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.564689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] RBAC: Adding query filter params (type=network) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:33.564984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:33.566370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:33.578939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 386/780] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:33.589465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.627506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:33.628443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.628911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.629369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.654590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:33.659307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:33.661184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:33.662830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 395/781] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:33.667857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.778828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0826a-9a9c-4a6e-aa52-d0a533c24a48 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:33.824838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6f60-4f9d-4025-bfd0-25d92893901b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:33.842223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:33.842776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.843051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.843318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.844033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] RBAC: Adding query filter params (interface=public, service_id=056d8d1713254911bf958ffd0170da14, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:33.844334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:33.845699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:33.864852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 387/782] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/endpoints?service_id=056d8d1713254911bf958ffd0170da14&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:33.872107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:33.877047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:33.894445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:33.895064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:33.895607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:33.895872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:33.896560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:33.896860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:33.899088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:33.917226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 396/783] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:33.929514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:33.940255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:34.048271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1958d3-b2ec-40c0-862d-3cf91d1df09a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:34.113076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684c80-51cb-4ad7-af05-8e3ddaf9d439 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:34.117567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:34.118238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.118597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.118774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.124750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:34.126202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:34.138536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 388/784] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/services => generated 772 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:34.163622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:34.164262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.164532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.164799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.177692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:34.179290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:34.194226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:34.196575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:34.198704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:34.198704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:34.198704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:34.198704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 397/785] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:34.201605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:34.328867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:34.334778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1117a-de6c-43e7-a594-8d9e52c8c15c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:34.380740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:34.381267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.381550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.381812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.382505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:34.382813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:34.384243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:34.405085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 389/786] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:34.411166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:34.457428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:34.515682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc6fdc-7d2c-4925-a692-c8b7f8d2da50 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:34.549761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:34.550494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.551195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.551462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.559535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:34.560991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:34.572732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:34.573751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:34.574771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:34.578428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 398/787] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:34.580309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:34.701533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3e919-c990-44a4-87d0-4073419805d0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:34.742838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:34.743510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.743786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.744053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.744778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:34.745084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:34.746540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d9304-70f7-475d-b1b9-1f2c4fae863e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:34.773552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 390/788] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 9 02:28:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:34.779053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:34.810602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493385-0435-4882-9be4-fa1b22a3970e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.810886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493385-0435-4882-9be4-fa1b22a3970e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.811213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493385-0435-4882-9be4-fa1b22a3970e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.812637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 391/789] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:34.835711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.836059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.836243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.864375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:34.865579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:34.902524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:34.952239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8672c-7ed8-426f-884c-06e2cea00f08 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:34.953966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:34.964228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 392/790] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:34.994254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:34.994900 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:34.995180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:34.995454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:34.998380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.006869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.008349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:35.023184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:35.024198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:35.025173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:35.025173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:35.025173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:35.025173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 399/791] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:35.034125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.188470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3be61a-562f-4082-8191-f2bd77c98243 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.191956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae23312-3f8d-4d78-9891-6a46efe6fa47 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.228321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.228850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.229126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.229395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.230041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:35.230358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.231932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112690-0ede-4f6d-8c22-ffcb22517733 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.244882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 400/792] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:35.248608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.249528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.249788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa4a4-dce3-4b56-9892-fdc74d8b6828 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.250074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa4a4-dce3-4b56-9892-fdc74d8b6828 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.250366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60aa4a4-dce3-4b56-9892-fdc74d8b6828 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.250701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.251190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.251829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 401/793] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:35.252189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.253465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa4a4-dce3-4b56-9892-fdc74d8b6828 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.254095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-9c91228f-833d-4729-bb99-d8732ab64c0d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:35.267423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:35.268505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 02:28:35.269370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:35.270125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:35.272446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 393/794] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 9 02:28:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:35.274415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.274783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.275304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.293751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:35.295289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:35.303904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:35.361918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60aa4a4-dce3-4b56-9892-fdc74d8b6828 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.393882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 394/795] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:35.395143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.397239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.397882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.398331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.398550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.410635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.410635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:35.426350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:35.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 402/796] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:35.428747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.518671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:35.527947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23af44-06ef-4ddd-bee0-5dd4d1f7472b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.536213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85da354-53b5-44b0-8a65-637414bbe432 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.570734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.571500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.571785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.572019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.572721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:35.573020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.574486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.586989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 403/797] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:35.587793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.593012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.593647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.593930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.594186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.595073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:35.596263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.597659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e9812-e44d-405e-ae84-2b169c92ab89 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.608118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 395/798] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:35.608118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188714-9fe8-4975-bf21-588a32879528 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.608118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188714-9fe8-4975-bf21-588a32879528 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.608118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188714-9fe8-4975-bf21-588a32879528 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.608118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 396/799] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:35.608118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188714-9fe8-4975-bf21-588a32879528 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.618109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:35.700210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238481b-0483-485b-8930-3aaee62aa560 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.729796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188714-9fe8-4975-bf21-588a32879528 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.733996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.734655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.734933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.735197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.737511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:35.748562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:35.753436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:35.754530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:35.755420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:35.758500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 404/800] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:35.760096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.768153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a2ac-7478-457f-b208-1c1e66023a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.768881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a2ac-7478-457f-b208-1c1e66023a7c None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.769101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a2ac-7478-457f-b208-1c1e66023a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.769841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a2ac-7478-457f-b208-1c1e66023a7c None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/5ea3e582aac94931b1a041817ccab685/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.772208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a2ac-7478-457f-b208-1c1e66023a7c None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=5ea3e582aac94931b1a041817ccab685, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.774260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a2ac-7478-457f-b208-1c1e66023a7c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.792522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 397/801] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:35 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/5ea3e582aac94931b1a041817ccab685/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:35.793675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.793970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.794263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.814357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:35.815876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:35.902266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:35.904398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a28ab-97fd-48dd-8adc-9df8d4b0aa98 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:35.918607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 398/802] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:35.922020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:35.957944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:35.959273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:35.959742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:35.960229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:35.960950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:35.961325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:35.963143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:35.968708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 405/803] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:35.970904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.042291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.063998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d493ea1-0f1e-496d-8acf-fdb77c4c5779 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.088670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44727d2d-e02b-4fef-b98b-cc14a65e80e7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.109264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.109895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.110198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.110994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.111811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:36.112255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.114323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.125429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.125623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 399/804] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:36.126285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.126630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.126963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.130419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.141630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:36.146858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:36.147788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:36.148650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 406/805] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:36.149348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.204220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.263229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5930a-1db7-4593-83c9-d583cfd799fa None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.266546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbf788-965a-4e63-b032-11a3e675a401 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.284603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.305901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.306458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.306774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.307046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.314828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.316128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.316638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.317356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.317855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.318428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.319484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.320206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:36.335282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:36.336679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:36.338427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:36.338427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:36.338427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:36.342227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 407/806] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:36.343348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.350687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 400/807] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:28:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:36.351839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.482543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c8f88-41b7-4434-a825-6205747401aa None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.483720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2afa0-7708-4ff8-ae9c-dd5d8b978ca2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.495850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.524618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.525400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.525681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.526018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.526786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:36.527156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.528646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.539507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.539719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.539827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.542221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.542221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:36.542221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.542711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.549811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 401/808] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:36.559373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.572773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 408/809] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:36.578165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.632407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.692135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b9adb-f911-43da-a69a-a0da822ab1d4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.725300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db33c8-2261-44fe-a414-517c1ebe2265 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.738587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.739314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.739498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.739773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.749787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.751296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.761777 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.762442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.762717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.762984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6e2b22ac-d5af-45b7-a964-0dd5bc049f7c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:36.765017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:36.766035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:36.767044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 402/810] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:36.774187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a03cb-63d8-405a-8333-d770371d8848 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.774187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a03cb-63d8-405a-8333-d770371d8848 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.774187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a03cb-63d8-405a-8333-d770371d8848 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.774602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 403/811] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:36.780593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.780872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.781136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.784210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.785907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:36.801235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:36.802279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:36.803214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:36.803214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:36.803214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:36.803405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:36.803405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:36.805465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 409/812] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:36.806603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.835621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.897922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 404/813] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:36.897922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:36.907854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:36.933073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b5eb-60ad-4eaa-b09b-d1da0589846b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:36.968123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:36.969055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:36.969506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:36.969976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:36.970761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:36.971070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:36.972514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:36.999571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 410/814] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:37.001105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.016748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1350cd-c181-406d-971f-d2ab7c4750d0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.052797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.053419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.053698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.053968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.054638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:37.055038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.056489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.069544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 405/815] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:37.070572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.102786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.106733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a3b18-1dfe-470a-a404-4e7b8fd4aa77 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.145281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.145967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.146295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.146577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] PATH_INFO: `/v3/services/network` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.147181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.148582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:37.157550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:37.158599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 02:28:37.159463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:37.163958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 411/816] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:36 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:37.164809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.190523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.191534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155870-3862-40d2-84c5-f6b21f6ed1aa None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.233638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.234401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.235205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.235545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.252567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.254566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.269878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c417dec-cfa4-4ffb-adaf-28e85ca53577 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:37.272228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:37.273546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:37.274655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:37.274655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:37.274655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:37.274655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 406/817] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:37.278178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.311426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.312260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.312637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.313007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.321161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.322864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:37.334758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:37.335928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:37.337957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 412/818] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:37.339422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.358206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.396822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a1c0-b058-479b-b167-854a1bd068f8 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.408520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.437624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.438406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.439793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.440073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.440786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:37.441097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.442623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.452303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 407/819] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:37.452610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27228c46-ddde-4f8d-a0db-763da220ae06 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.453262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.494608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.495570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.496431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.496887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.497714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:37.498571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.500444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.519908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 413/820] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:37.519908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.565314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda1556-e09a-44ed-b31b-7b2825334fa9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.603213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.603803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.604092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.604369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.605118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:37.605425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.607095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.610053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.629478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 408/821] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:37.630364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.631099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3affc-ffa1-4e2d-a3c7-14f0a7c10f69 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.662775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.672193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.673149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.673637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.674105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.674976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] RBAC: Adding query filter params (type=network) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:37.675479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.677062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.687039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 414/822] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:37.688156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.741809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fda472-2cbe-4d2c-a7b1-ea2608710dd2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.781931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.782624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.782952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.783176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.790757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.791594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac59d28-1b29-4ec5-8953-fa92444cd646 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.792217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:37.804655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:37.805589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:37.806606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:37.810584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 409/823] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:37.811421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:37.830685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094b86-347c-4084-8f33-49e881254405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.831681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094b86-347c-4084-8f33-49e881254405 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.832183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094b86-347c-4084-8f33-49e881254405 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.832653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094b86-347c-4084-8f33-49e881254405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.872514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.891319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:37.902130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094b86-347c-4084-8f33-49e881254405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.904244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094b86-347c-4084-8f33-49e881254405 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.931990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3a33a-8df1-4982-a286-1a83dfc289a0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:37.969089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:37.969939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:37.970396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:37.970816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:37.971905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:37.973629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:37.995624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 410/824] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:28:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:37.996279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.014073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 415/825] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:28:38.016544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5559-b056-4d4e-8cd4-686cc84b2d1a None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.017008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5559-b056-4d4e-8cd4-686cc84b2d1a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.017434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5559-b056-4d4e-8cd4-686cc84b2d1a None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.020838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 416/826] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:38.022103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5559-b056-4d4e-8cd4-686cc84b2d1a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.149756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfdd2f-7dd3-472c-ac1e-e526d2566ad2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.151489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.161672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.165567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5559-b056-4d4e-8cd4-686cc84b2d1a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.202531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.202531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.202531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.202531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.202949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:38.202949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.203787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02eda-8f82-4a8f-ab84-ce7f30cb888a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.217069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 411/827] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:38.218587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.219008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.219427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.222775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.223429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.223706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.224281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.238481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.238974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:38.240165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:38.240266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:38.263459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:38.264622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:38.265571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:38.265571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:38.265571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:38.268053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 417/828] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:38.269755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.314115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 412/829] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:38.314989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.388007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89e9dd-9eee-4f9b-b83e-62b39edf322b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.411929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.419975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338bc2a-55fa-433c-9ebe-28f4d15d835e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.427867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.428540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.428862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.429100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/562c6203e9cf440ebd81d14877c9e452/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.441724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.446697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=562c6203e9cf440ebd81d14877c9e452, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.449033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.456275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.457021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.457356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.457686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.458530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:38.458909 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.460308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.473534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 418/830] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:38 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/562c6203e9cf440ebd81d14877c9e452/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:38.473534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.477216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 413/831] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:38.478317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.594477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1d939-604a-4583-ac89-86b4b1abd210 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.615824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c263a-9b0f-4b0a-a4a5-d090b0b02b83 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.634095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.634564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.634841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.635108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.644314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.645782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.659297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef731c9-c87e-4a20-b24b-b71210dad1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.659944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef731c9-c87e-4a20-b24b-b71210dad1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.660229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef731c9-c87e-4a20-b24b-b71210dad1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:38.660488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:38.661348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:38.662689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:38.663361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef731c9-c87e-4a20-b24b-b71210dad1d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.663443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 414/832] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:38.667712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.672402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef731c9-c87e-4a20-b24b-b71210dad1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-4ef731c9-c87e-4a20-b24b-b71210dad1d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:38.679637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:38.680634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:38.681651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 419/833] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:38.683089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7a19-85db-4912-a4a7-a98baecc8937 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.683385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7a19-85db-4912-a4a7-a98baecc8937 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.683850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7a19-85db-4912-a4a7-a98baecc8937 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.686271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 420/834] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:38.688091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c7a19-85db-4912-a4a7-a98baecc8937 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.691431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.745189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.777760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e866e-7a5e-4c9a-8c90-bda15f00693f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.796170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c7a19-85db-4912-a4a7-a98baecc8937 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.810824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.811441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.811727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.812001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.812701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:38.813007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.814484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0843c-63ea-4a3a-b435-ebf48d159e14 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.826695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 415/835] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:38.827969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.828259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.828534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.844727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:38.845604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:38.846075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:38.846553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:38.847395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:38.847865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:38.847986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:38.849826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:38.850141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:38.865368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 421/836] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:38.865368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.925061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 416/837] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:38.926339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:38.949045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:38.978279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18be19-00bb-46e5-8878-ed4f469bb29f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:38.997050 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:39.020226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.020810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.021089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.021354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.022086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=ad4a7ffff8f842969d782603cbb8fb89) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:39.022432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.024002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.034353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 422/838] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/role_assignments?user.id=ad4a7ffff8f842969d782603cbb8fb89&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:39.035262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.045765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537f025-bca3-4e99-abba-10efe9b1a965 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.083027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.083700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.083983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.084257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.098146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.100077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:39.119575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:39.120662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:39.121594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:39.121594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:39.121594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:39.121594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 417/839] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:39.126765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.188935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd4c5-d050-478e-88c5-65f849257547 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.191793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:39.237059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.238366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.239044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.239463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.240386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.242599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.244446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cd732-0bce-4930-9167-ba898765ac96 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:39.254090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:39.255260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:39.256104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:39.257246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 423/840] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:39.264532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.296625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.297567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.297898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.298202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.298966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:39.299265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.300688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.318015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 418/841] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:39.324032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.338984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:39.424202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3006e-ea54-4c44-920d-fe33784a5df9 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.460927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.461897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.462462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddf6dd-5856-4e54-8d9a-9bee9f51676a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.462736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.463398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.464249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:39.464693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.466292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.478269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 424/842] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:39.482880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.494049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:39.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.518861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.519639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.520018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.529568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.531614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:39.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:39.548764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 02:28:39.549968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:39.556092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 419/843] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:39.558309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.611175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf24b26-7603-4834-84e7-d55d3789d42b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.621496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:39.653702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.654497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.654847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.655304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.655902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:39.656353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.657814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.663046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 425/844] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:39.681760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c13b5-f8e7-47f9-83c2-60599077d83e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.721879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.723300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.723997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.725014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.726614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:39.727283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.729934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:39.752547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 420/845] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:39.758025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:39.866116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1abed2-92e1-425d-9629-973285bc0ee1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:39.901621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:39.902524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:39.902956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:39.903412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:39.955326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:39.986107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:39.987503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:40.090575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 426/846] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 02:28:40.096198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.131199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:40.198566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb44f2-26cc-4024-9699-ba029b4d0d8e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:40.235074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:40.235955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.236436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.236850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.247542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:40.249738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:40.270654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:40.271726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:40.272859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:40.272859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:40.272859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:40.272859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 421/847] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:40.280563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.353336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:40.385711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccfa16-759b-4ac0-a179-700139511260 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:40.421235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:40.422324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.422798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.423260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.424156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:40.424667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:40.426758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bc4ed-b166-4702-a55f-2508181fb4a6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:40.443725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 427/848] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:40.447584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.542449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:40.554196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465eeb-ec8a-4ff6-a9d0-f8f7abbafc53 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:40.585080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d753b-d7e3-4944-b915-0e0f6b6afd84 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.585372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d753b-d7e3-4944-b915-0e0f6b6afd84 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.585652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d753b-d7e3-4944-b915-0e0f6b6afd84 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.587362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 428/849] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:40.589338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:40.590219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.590670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.591106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.592240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.592593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.592935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.598801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:40.600814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:40.611710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:40.612467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:40.613338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:40.614370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:40.614802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:40.616437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 422/850] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:40.623061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.687466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 429/851] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:40.709189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.747927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65864475-a9e8-487f-b512-2eb5cdb0a8cd None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:40.782611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:40.792593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:40.793526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.794023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.794537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.795436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:40.795965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:40.797706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:40.811392 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 423/852] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:40.817698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.828657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:40.837745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e4348-757b-4e25-8686-711cd2c4b6f6 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:40.875820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:40.876922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.877403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.877874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.889322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:40.894631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:40.895593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:28:40.896487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:40.904110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 430/853] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:40.905794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:40.921380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f655adf-411d-4fa6-8201-4d93350740b1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:40.958621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:40.959239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:40.959528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:40.960125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:40.960594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=5ea3e582aac94931b1a041817ccab685) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:40.960899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:40.962381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:40.974594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 424/854] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/role_assignments?user.id=5ea3e582aac94931b1a041817ccab685&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:41.020432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72709b-9735-49b8-bdb3-0dbec8b74585 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.045675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:41.061350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.061976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.062286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.062562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.063248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:41.063553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.065109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21259895-765e-4b03-8f16-a4ead41c8ab3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.075035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 431/855] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:28:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:41.080688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.157771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:41.163874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5a636-945f-49c3-9da5-ea65e4067a44 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.164437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5a636-945f-49c3-9da5-ea65e4067a44 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.164871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5a636-945f-49c3-9da5-ea65e4067a44 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.166535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 432/856] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:41.172154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.172647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.172925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.191205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31cc0-452e-4c4c-95c0-6f0bd16b47d0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.194843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:41.196046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:41.226463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.227097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.227374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.227922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.238192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.239688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:41.256114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:41.257758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:41.258733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:41.258733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:41.258733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:41.262556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 425/857] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:41.264138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.279356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 433/858] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:41.300732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.368539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f593-0ba4-49cd-b604-3114fb9c234f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.393084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:41.415287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.415986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.416360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.416699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.417422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:41.417788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.418127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13933f4c-267f-49b3-8bd7-278501b9d521 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.419384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.437048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:41.437630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 426/859] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:41.438700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.455450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.456354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.456730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.457116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.457879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.459387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:41.469035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:41.470142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:41.471237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:41.472119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 434/860] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:41.478084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.561419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5fb20-aca4-4c48-9de0-38d65d1ec140 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.582933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f223aa-a357-4c9e-bed2-8a74be519a35 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.605822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.606798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.607466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.607972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.617346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.617875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.618682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.618858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.619145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.619531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.620208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:41.620586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.622004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.631973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 435/861] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:41.632535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:41.633484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:41.634556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:41.635102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.635174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 427/862] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:41.639915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.697687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:41.698558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:41.747047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796d008-0aa9-495e-b754-17e24b475733 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.775614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d750670-a26a-4da7-8d57-af11bc03a730 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.789571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.790626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.791312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.792275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.793415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:41.793929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.795937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2643b-9c95-4050-abc0-80c2fc4137ab None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.802709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 436/863] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:41.824715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:41.825320 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.825600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.825871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.826696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:41.827004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:41.828461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:41.845378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213a0f-fec7-4b49-9edd-c892c6e3a4f4 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.846020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213a0f-fec7-4b49-9edd-c892c6e3a4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.846539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213a0f-fec7-4b49-9edd-c892c6e3a4f4 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.848392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 437/864] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:41.852576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 428/865] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:41.854382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:41.854580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:41.854864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:41.856099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.873852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:41.875097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:41.958320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 438/866] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:41.983133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:41.990842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698cae5-9de7-4de5-8999-7fcfde56ef12 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:41.993322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:42.024997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:42.030578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:42.031727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:42.032181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:42.032649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:42.110829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a70d6-348f-45e7-8d95-8114cff061d2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:42.128486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:42.130213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:42.152680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:42.153339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:42.153624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:42.153908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:42.154610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:42.156577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:42.178828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 439/867] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:28:41 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:28:42.253463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 429/868] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:28:42.259377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:42.381778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:42.399253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf3920-7e26-4463-b00c-409349251bdb None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:42.440629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:42.442000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:42.442496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:42.442975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:42.454046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:42.455818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:42.470310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:42.471314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:42.472234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:42.472234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:42.472234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:42.474656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 440/869] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:42.482375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:42.483100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:42.584729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f17f5-d300-4a5a-b64a-13cca161c647 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:42.619968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:42.620854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:42.621296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:42.621713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:42.622538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:42.622984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:42.624488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:42.637213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 430/870] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:42.642994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:42.746927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39825ea8-9e13-466c-8d24-275865060f53 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:42.785283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:42.785922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:42.786223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:42.786494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:42.793393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:42.794884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:42.806253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:42.807189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:42.808108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 441/871] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:42.815026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:42.889454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:42.930629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc907-62a2-4309-9bc2-b6cc64bd07c0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:42.968995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:42.970017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:42.970559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:42.971029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:42.971924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:42.972441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:42.974559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f7ecc-3ca9-475b-942a-6f9e3d5563eb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:42.987776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 431/872] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:42.994111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:42.997672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:43.101072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b6f3e-f926-41df-96d1-d5637fa54879 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:43.104966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f2922-be8e-4b79-912b-5cf18b632ed2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.105257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f2922-be8e-4b79-912b-5cf18b632ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.105531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f2922-be8e-4b79-912b-5cf18b632ed2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.107165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 432/873] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:43.111939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.112226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.112504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.130914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:43.132213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:43.138470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:43.139371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.139877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.140333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.141298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=562c6203e9cf440ebd81d14877c9e452) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:43.141783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:43.143411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:43.149909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 442/874] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:42 2022] GET /identity/v3/role_assignments?user.id=562c6203e9cf440ebd81d14877c9e452&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:43.204815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 433/875] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:43.228011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:43.304958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:43.343226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e9f99-6513-4c20-b081-58e783dce1f4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:43.380495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:43.381507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.381801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.382055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.382696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:43.384140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:43.394624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:43.395639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:43.396506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 443/876] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:43.401995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:43.461262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:43.515251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd9ff-4bbf-4a53-9558-8061df5c6a89 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:43.550509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:43.551385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.551844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.552267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.553044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:43.553508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:43.555045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eff13d-3d83-42f6-a878-94e9960fd0ff None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:43.560320 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 434/877] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:43.571329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:43.676940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ea09-be33-4d07-934d-68d0c2dce67c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:43.706853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:43.713938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:43.714909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.715431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.715904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.716749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:43.717254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:43.718884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf41d8-7456-476b-abd7-63e62eeea30d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:43.730014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 444/878] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:43.851306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c8ea-42a5-4c76-9d15-f786a836ee21 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.851637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c8ea-42a5-4c76-9d15-f786a836ee21 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.851852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c8ea-42a5-4c76-9d15-f786a836ee21 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.853584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 435/879] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:43.859283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ab50e-01f1-4f68-a009-0f0f9efdf57d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:43.859589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ab50e-01f1-4f68-a009-0f0f9efdf57d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:43.859839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ab50e-01f1-4f68-a009-0f0f9efdf57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:43.880012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ce5ab50e-01f1-4f68-a009-0f0f9efdf57d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:43.881207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ce5ab50e-01f1-4f68-a009-0f0f9efdf57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:43.954140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 445/880] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:43.982302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c8ea-42a5-4c76-9d15-f786a836ee21 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:44.085904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c8ea-42a5-4c76-9d15-f786a836ee21 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:44.111046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:44.123015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026d2ee-ecb1-4161-b1f2-ad6830c0803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:44.123636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026d2ee-ecb1-4161-b1f2-ad6830c0803f None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:44.123920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026d2ee-ecb1-4161-b1f2-ad6830c0803f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:44.124197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026d2ee-ecb1-4161-b1f2-ad6830c0803f None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:44.124850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026d2ee-ecb1-4161-b1f2-ad6830c0803f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:44.126257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026d2ee-ecb1-4161-b1f2-ad6830c0803f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:44.144444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 436/881] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:28:43 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:44.275240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a607-5660-45ac-bee7-3e03e9d4d2ce None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:44.275704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a607-5660-45ac-bee7-3e03e9d4d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:44.276092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a607-5660-45ac-bee7-3e03e9d4d2ce None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:44.277959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 446/882] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:44.283244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:44.283531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:44.283806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:44.304942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:44.306282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:44.393899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 437/883] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:44.415736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a607-5660-45ac-bee7-3e03e9d4d2ce None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:44.525093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a607-5660-45ac-bee7-3e03e9d4d2ce None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:44.550144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:44.562075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:44.563066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:44.563583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:44.564055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:44.564844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:44.566446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:44.576985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:44.578450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:44.579333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:44.582053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 447/884] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:44.594174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:44.689344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700edde-fe58-477a-9f70-e9ec249705ab None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:44.690857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:44.724703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:44.725325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:44.725610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:44.725871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:44.726525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:44.726826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:44.728284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:44.733529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 438/885] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:44.741994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:44.841881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e662f-04d8-42f4-b7ab-29902f69620f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:44.877592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:44.878236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:44.878522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:44.878795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:44.879436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:44.879739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:44.881113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:44.886556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 448/886] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:44.892045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:44.994230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3672-e11f-4103-b228-34998d9bb4ac None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:45.027584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:45.028240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.028517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.028831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.029529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] RBAC: Adding query filter params (interface=public, service_id=eb4d509507e1410c8da3b7e5d9aee275, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:45.029830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:45.031205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f77bb3-6e04-4ce6-87ac-357a0b29d871 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:45.036493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 439/887] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:28:44 2022] GET /identity/v3/endpoints?service_id=eb4d509507e1410c8da3b7e5d9aee275&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:45.042088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:45.098202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:45.148732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7035e5-6b1d-4bea-bcb7-02abfb76aa98 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:45.181841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94496f-394f-45e1-b4c8-32091a5377ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:45.182521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94496f-394f-45e1-b4c8-32091a5377ac None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.182797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94496f-394f-45e1-b4c8-32091a5377ac None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.183065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94496f-394f-45e1-b4c8-32091a5377ac None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.183673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94496f-394f-45e1-b4c8-32091a5377ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:45.185033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94496f-394f-45e1-b4c8-32091a5377ac None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:45.191217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 449/888] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:45 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:28:45.452245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cbabc-203c-46d9-a569-c2b771fb8904 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.452245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cbabc-203c-46d9-a569-c2b771fb8904 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.452245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cbabc-203c-46d9-a569-c2b771fb8904 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.453143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 440/889] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:45.458601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.459149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.459637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.478683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:45.480202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:45.549482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 450/890] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:45.572574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cbabc-203c-46d9-a569-c2b771fb8904 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:45.604133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:45.677499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cbabc-203c-46d9-a569-c2b771fb8904 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:45.713483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:45.714414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.714414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.714736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.726180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:45.731043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:45.731998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:45.732864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 441/891] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 9 02:28:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:45.740068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:45.787259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:45.844008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cf14a-a4f7-4e35-9dd6-4708c3f09c1d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:45.878803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:45.879420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.879715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.879965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.880639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:45.880962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:45.882416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0dee5-155c-40dc-ab5f-c9ed19feef8e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:45.891503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 451/892] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:28:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:45.900235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:45.901352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a355b-8aa8-449e-8755-898a7f9e8353 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.901648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a355b-8aa8-449e-8755-898a7f9e8353 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.901899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a355b-8aa8-449e-8755-898a7f9e8353 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.903553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 452/893] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:45.909920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:45.909920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:45.909920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:45.931143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:45.932851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:46.006721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.018605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8018-4a0a-4889-9e3d-308d90421201 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.030309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 453/894] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:46.051385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.068031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.069037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.069525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.069993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.081372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.083159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.102665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:46.104316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:46.105255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:46.106218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:46.106218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:46.106218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:46.111989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 442/895] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:46.116740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.169471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce1a99-32da-4527-a5c2-e9c5d1fe91f2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.213615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.214186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.214740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.215020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.215677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.217800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.226195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcc0dd-1276-4a82-9c1a-a96fd35810ad None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.249840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 454/896] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:28:46 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:28:46.281081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.282021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.282517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.282970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.283781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:46.284262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.285952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca40798-0afe-494a-84b7-ef575a496bc2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.294856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.302308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 443/897] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:46.311789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.365479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8773509-63c9-4f0b-a490-269b000919dc None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.366197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8773509-63c9-4f0b-a490-269b000919dc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.366681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8773509-63c9-4f0b-a490-269b000919dc None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.368671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 444/898] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:46.374729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.375315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.375830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.395488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:46.397044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:46.440420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.445514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c082b9-5eac-4e06-98af-f9ad6cf17315 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.488988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 445/899] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:46.492372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.493036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.493471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.493750 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.503336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.504789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.512776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:46.516631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:46.517799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:46.518920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 455/900] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:46.524546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.631677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c09e3-d948-49b6-a3c2-b51965fe235a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.660612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4954-2396-48b9-a9f3-a9e9412e6943 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.667370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.667992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.668271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.668542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.669283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:46.669589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.671178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.691440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 456/901] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:46.697169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.704689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.717993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.718628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.720571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.721115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.721654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.723263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.725928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:46.739789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:46.740758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:46.741682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:46.750192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 446/902] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:46.752665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.830701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741f590-b562-4a7a-b882-fc7996f7b654 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.861416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7ac88-afff-4b57-a23d-296a90faa57d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:46.875284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.876585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.876950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.898891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:46.899837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:46.900287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:46.900699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:46.901615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:46.902058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.903848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.909538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 447/903] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:46.914815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:46.971579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:46.973339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:46.979528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:46.994145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.044536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d5533-2d36-4b8e-9d85-bb5df0c4b672 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.092816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.093973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.094484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.095304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.096418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:47.096950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.098547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.102298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 457/904] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:28:47.105975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.112768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 448/905] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:47.120109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.229096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1485c-f1d5-4655-80a4-8082e5520733 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.233355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cdfe1-cc91-486f-aa61-6e66e5584534 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.246380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.257985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.267766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.268699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.269188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.269884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.275712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.276324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.276600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.276867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.277572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] RBAC: Adding query filter params (interface=public, service_id=44e68d5871354c56acf9672e7bee538d, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:47.278328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.279699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415f1d9-b05c-4ea1-8b99-c7b07255ce4e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.281708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.283348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.284812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 449/906] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/endpoints?service_id=44e68d5871354c56acf9672e7bee538d&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:47.286190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a31b-eb27-44d2-bc72-263c673faabf None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.286190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a31b-eb27-44d2-bc72-263c673faabf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.286593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a31b-eb27-44d2-bc72-263c673faabf None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.287784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 450/907] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:47.292883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8a31b-eb27-44d2-bc72-263c673faabf None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ff961a02-222b-40f0-b346-5f5737b4034d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:47.301189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:47.302200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:47.303152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:47.303152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:47.303152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:47.303818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 458/908] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:47.304562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.304891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.305169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.323803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:47.325001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:47.402182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 459/909] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:47.402182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.410649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8a31b-eb27-44d2-bc72-263c673faabf None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.449227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.449876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.450141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.450431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.451033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.452401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.466182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 451/910] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:28:47.466182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.515860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.533481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.536279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c592f4-6c84-4474-b1b4-6c31e7edc2b4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.578011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99801be6-49c3-45af-a813-1248170c0e6a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.591425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.592445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.592832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.593223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.594122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:47.594925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.596955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.614482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 460/911] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:47.619992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.622467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.623238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.623614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.624007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.625063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.627682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:47.639307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:47.640595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:47.641663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 452/912] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:47.650250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.738376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013122ad-5410-40f6-b219-edfaedb2691f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.775028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.775910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.776260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.776685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.783169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570fbf6-6f3c-4edf-a00e-74704e4d404e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.784007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.785723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.798116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:47.798681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:47.799586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:47.800570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:47.803714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 461/913] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:47.814214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.820994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.821745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.822575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.822864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.823517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:47.824221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.825890 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.831009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:47.834655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 453/914] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:47.839464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:47.918515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268044a5-17c6-40ac-93c0-07ce0ab81ed1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.960235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:47.960864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:47.961144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:47.961417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:47.962252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:47.962519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:47.963963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:47.969197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f202564-3550-4dd7-a7a7-3bdf095b47fc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:47.981940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 462/915] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:47.982909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:48.012465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:48.013751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.014456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.014887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.015907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:48.016354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:48.018491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:48.025469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 454/916] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:48.033849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:48.079467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:48.085462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:48.107467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f785-3c3c-4a73-bf4a-bcb27fb91aa6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:48.149454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:48.150186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.150705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.150990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.151748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] RBAC: Adding query filter params (role.id=9e89c752b6b94dcdbdc07bb30bc586db, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=562c6203e9cf440ebd81d14877c9e452) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:48.152064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:48.153447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdd753-d41c-4274-b59a-b968e1f4f5bd None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:48.153562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fd344-b098-44b0-96f1-c39746ed9455 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:48.162304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 463/917] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:47 2022] GET /identity/v3/role_assignments?user.id=562c6203e9cf440ebd81d14877c9e452&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=9e89c752b6b94dcdbdc07bb30bc586db => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:48.196877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4094125-e1be-4877-acf7-42306e6ad3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:48.197973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4094125-e1be-4877-acf7-42306e6ad3de None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.198481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4094125-e1be-4877-acf7-42306e6ad3de None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.198952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4094125-e1be-4877-acf7-42306e6ad3de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.199872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4094125-e1be-4877-acf7-42306e6ad3de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:48.201587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4094125-e1be-4877-acf7-42306e6ad3de None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:48.227395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 455/918] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:28:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:48.374532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0a85-2417-4703-adcd-b8468a1eabbb None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.375317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0a85-2417-4703-adcd-b8468a1eabbb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.375802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0a85-2417-4703-adcd-b8468a1eabbb None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.377807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 464/919] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:48.383271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.383560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.383839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.406230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:48.407463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:48.484671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 456/920] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:48.506518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0a85-2417-4703-adcd-b8468a1eabbb None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:48.611146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0a85-2417-4703-adcd-b8468a1eabbb None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:48.646185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:48.646804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.647078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.647393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.647985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:48.649353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:48.658685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:48.659700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:48.660558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 465/921] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:48.668117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:48.695447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:48.779812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa65f5-f70c-4083-aabe-9757aeda7c59 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:48.814434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:48.815098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.815377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.815683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.816312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:48.816610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:48.817973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:48.823306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 457/922] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:48.831278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:48.841937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:48.935056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639df9a5-32b4-4fa0-a5aa-1860d117b5d4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:48.969537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:48.970577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:48.971277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:48.971727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:48.972543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:48.973023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:48.974972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:48.985445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 466/923] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:48.987069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:49.075387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:49.088993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c00bdd-d97e-4a1b-b48f-b8d8689e20d3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:49.127464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:49.128451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:49.128927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:49.129395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:49.130328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] RBAC: Adding query filter params (interface=public, service_id=c8eceff23f24449fba22129c0e1d9543, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:49.130854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:49.132941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423df9f-af4b-470a-89f5-16c7d5efc7bb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:49.144011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 458/924] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:28:48 2022] GET /identity/v3/endpoints?service_id=c8eceff23f24449fba22129c0e1d9543&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:49.145769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:49.225933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:49.250754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82330b1-6dff-4eb1-ba44-c9d010124f5c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:49.285442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944379fb-fd06-47b4-9a26-cf49d60aabb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:49.286059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944379fb-fd06-47b4-9a26-cf49d60aabb7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:49.286368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944379fb-fd06-47b4-9a26-cf49d60aabb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:49.286641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944379fb-fd06-47b4-9a26-cf49d60aabb7 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:49.287254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944379fb-fd06-47b4-9a26-cf49d60aabb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:49.288636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944379fb-fd06-47b4-9a26-cf49d60aabb7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:49.302355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 467/925] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:49 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:28:49.635065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966d296-02ec-4f37-aaf5-7ec500bf679f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:49.635457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966d296-02ec-4f37-aaf5-7ec500bf679f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:49.635639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966d296-02ec-4f37-aaf5-7ec500bf679f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:49.637266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 459/926] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:49.642517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:49.642805 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:49.643075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:49.661250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:49.662453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:49.733309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 468/927] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:49.758130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966d296-02ec-4f37-aaf5-7ec500bf679f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:49.862232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966d296-02ec-4f37-aaf5-7ec500bf679f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:49.896439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:49.897003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:49.897281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:49.897604 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:49.898249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:49.899612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:49.908467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:49.909424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:49.910386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 460/928] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 9 02:28:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:49.915924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:49.982820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.020582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757e7dd-a357-4abc-b5cf-7a7b5a80c5a5 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.055317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.056015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.056296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.056569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.057214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:50.057516 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.058936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a957b5-fe71-40de-9247-d7413779b433 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.064446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 469/929] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:50.071661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.088151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.175978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1933e23-ab15-4fa5-942e-2bd124a219fb None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.194732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaada10-7ebf-4fec-bf52-ad9d9e21a480 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.195027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaada10-7ebf-4fec-bf52-ad9d9e21a480 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.195284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaada10-7ebf-4fec-bf52-ad9d9e21a480 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.196921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 470/930] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:50.201658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.201934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.202224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.209746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.211227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.211508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.211780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.212532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:50.212836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.214224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.220093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 461/931] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:50.221958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:50.223231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:50.226171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.303207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 471/932] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:50.323050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.328801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.352843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48197-01cc-4284-a032-d9c79b7c9dcc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.388051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a87d-7ca1-4775-8578-7c0bf12dcc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.388656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a87d-7ca1-4775-8578-7c0bf12dcc50 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.388931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a87d-7ca1-4775-8578-7c0bf12dcc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.389197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a87d-7ca1-4775-8578-7c0bf12dcc50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.389874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a87d-7ca1-4775-8578-7c0bf12dcc50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.391278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a87d-7ca1-4775-8578-7c0bf12dcc50 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.396114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.412327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 462/933] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:28:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:28:50.413968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2b803-0750-407d-9be6-934f64769bf1 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.414507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2b803-0750-407d-9be6-934f64769bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.415024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2b803-0750-407d-9be6-934f64769bf1 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.416604 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 463/934] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:50.422655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.423163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.424207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.451220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:50.453988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:50.454962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89912b86-77da-470e-9960-b7e3e49c55ff None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.514372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.515797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.516495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.516921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.533013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:50.539417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:50.546551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:28:50.547571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.548228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 464/935] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:50.548306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 472/936] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:50.549858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.569996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.625254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.678587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc09bbd-803d-4b5e-9ad1-b22721309945 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.683520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44ed3-bf0c-43d5-aedc-15000cf0f630 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.714653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.718679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.719387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.719743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.720146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.720837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.721967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.722451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.723090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.723596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.724097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.725322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:50.725629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.725912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING keystone.server.flask.common [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 02:28:50.727384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.739285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 465/937] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:50.744560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.749760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.sql.core [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 02:28:50.749760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 02:28:50.749760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [parameters: {'id': 'f6de57c20c9744f6a744acc147ad8a6b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 02:28:50.749760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80497) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6de57c20c9744f6a744acc147ad8a6b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6de57c20c9744f6a744acc147ad8a6b. Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application conn.query(q) Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 02:28:50.766226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result.read() Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 02:28:50.767138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application next(self.gen) Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application yield resource Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application next(self.gen) Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise self.value Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application session.commit() Aug 09 02:28:50.768162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.session.flush() Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application compat.raise_( Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 02:28:50.769127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 02:28:50.770093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application conn.query(q) Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result.read() Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 02:28:50.772724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [parameters: {'id': 'f6de57c20c9744f6a744acc147ad8a6b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.774901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:50.781912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6de57c20c9744f6a744acc147ad8a6b. Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 473/938] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Aug 9 02:28:50 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:50.784386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.886637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12fe81-6117-4809-9d36-170268c30ad4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.895539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55756bed-d991-48cc-b92a-cc837d98fe9c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:50.922505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:50.934041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.934344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:50.934831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.935124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:50.935247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.935524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:50.935645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.935917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:50.943529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.945374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.948082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:50.949856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:50.958018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:50.958997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:50.959940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 474/939] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:50.966173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:50.967968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:50.969104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:50.969104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:50.969104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:50.969104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 466/940] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:50.969598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:50.972821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.005355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:51.084705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c56c2-586e-4ac2-bcd3-f9e821f4f394 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.097498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122953dd-a6dd-4240-bc79-8c80e68bbab1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.121162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.122109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.122625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.123089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.123951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:51.124509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.126225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.140237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.140910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.141193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.141468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.142123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:51.142488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.143842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.151931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 475/941] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:51.168974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 467/942] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:51.178217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.278455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:51.293479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f4488-9605-4adb-ae7f-d32b906ce5c2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.332107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.333157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.333642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.334114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.342501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.344239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-dec448d1-23d6-48e5-99a0-f2b8a558ec34 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:51.356825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:51.357893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:51.358931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:51.361535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 476/943] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:51.367309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.392901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1543a-2bf2-4d59-8893-f1534be71c1c None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.393206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1543a-2bf2-4d59-8893-f1534be71c1c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.393550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1543a-2bf2-4d59-8893-f1534be71c1c None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.395035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 477/944] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:51.402207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.402607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.402886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.424522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:51.429704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:51.431386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:51.491495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689725-b8da-4c2b-b108-45c4606d99cb None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.526289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 478/945] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:51.538179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.539054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.539517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.539985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.541057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:51.541587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.543224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.550866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.562510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 468/946] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:51.567793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.652645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:51.687431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1c17-fe17-4d05-ab89-70893df84aca None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.688348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:51.700884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66979c2a-b16f-4467-b31d-d1d0dfdf1867 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.723730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.724883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.725415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.725608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.726329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.727856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:51.737081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:51.738337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:51.739423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:51.741480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 479/947] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:51.747771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.748673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.748956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.749232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.751168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.761482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.763486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:51.784388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:51.785436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:51.787028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:51.787028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:51.787028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:51.787028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 469/948] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:51.791055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.877592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61973-6894-40af-a625-c58b4cf5dac0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.913029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.913615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.913898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.914191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.914834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:51.915140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.916551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.922189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 480/949] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:51.927592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:51.928318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76084f-6344-4a7a-9111-4333e44faf26 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:51.939165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:51.966046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:51.967150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:51.967634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:51.968147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:51.968998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:51.969502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:51.971283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:51.984910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 470/950] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:51.990409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:52.001753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:52.034303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58d43-bfe4-4736-99d0-7f74e8e1a183 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.068478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:52.069046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.069326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.069599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.070278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:52.070585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:52.072154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:52.081916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 481/951] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:52.083204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:52.099869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbda79-61e5-4bcb-b8a2-41bb12ee6528 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.136628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:52.137606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.138090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.138574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.146345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:52.148115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:52.156607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:52.161354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:52.162840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:52.163865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:52.165974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 471/952] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:52.171389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:52.193667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ae90-6008-48e8-83e3-3e090e501737 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.238318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:52.239350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.240347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.240824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.241945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:52.244200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:52.256140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:52.281636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 482/953] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:28:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:28:52.301792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0476-c46b-4db2-94d6-b643acbfd34e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.348998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:52.350180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.350565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.350931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.351735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:52.352136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:52.353751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a2528-90ec-4f33-9364-983e1585941d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:52.368468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 472/954] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:52.374278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:52.489413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b6a3-6274-413a-a1ce-c3c266e9e174 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.515470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:52.524290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb626003-4d40-482f-a05f-cb0342b04bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:52.525197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb626003-4d40-482f-a05f-cb0342b04bf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.525646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb626003-4d40-482f-a05f-cb0342b04bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.526122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb626003-4d40-482f-a05f-cb0342b04bf1 None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/562c6203e9cf440ebd81d14877c9e452/roles/9e89c752b6b94dcdbdc07bb30bc586db` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.528441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb626003-4d40-482f-a05f-cb0342b04bf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=562c6203e9cf440ebd81d14877c9e452, role_id=9e89c752b6b94dcdbdc07bb30bc586db)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:52.529969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb626003-4d40-482f-a05f-cb0342b04bf1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:52.546898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 483/955] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:28:52 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/562c6203e9cf440ebd81d14877c9e452/roles/9e89c752b6b94dcdbdc07bb30bc586db => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:28:52.566503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb75bb-6590-4f43-9a99-62ca6b6206d9 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.566886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb75bb-6590-4f43-9a99-62ca6b6206d9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.567236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb75bb-6590-4f43-9a99-62ca6b6206d9 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.569044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 473/956] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:52.575077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.575476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.575842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.596083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:52.597312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:52.672835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 484/957] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:52.695063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb75bb-6590-4f43-9a99-62ca6b6206d9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:52.804829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb75bb-6590-4f43-9a99-62ca6b6206d9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.828948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:52.840684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:52.841413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:52.841763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:52.842105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:52.842792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:52.844224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:52.853771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:52.855203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:28:52.856306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:52.862262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 474/958] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:52.864293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:52.967937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c46a5-8af3-4612-b6cd-cead6e5222e2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:52.986953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:53.003962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:53.004840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.005125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.005402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.006047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:53.006391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:53.008059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f1d3d-a1d2-4cb3-b33d-5ce7c03e07d2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:53.013563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 485/959] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:53.019317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:53.127791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4bc30-045e-40b4-b65b-34551110fcb9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:53.162626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:53.163597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.164055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.164519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.165370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:53.165851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:53.167417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f58a36-bf81-4b1b-a109-4f30fbdc2f29 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:53.172765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 475/960] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:53.321927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce773-5658-4d47-ad6d-3c1af2c0d1d9 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.322129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce773-5658-4d47-ad6d-3c1af2c0d1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.322422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bce773-5658-4d47-ad6d-3c1af2c0d1d9 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.324176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 486/961] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:53.329453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.329739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.330010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.348549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:53.350124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:53.398210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:53.428853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 476/962] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:53.452296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce773-5658-4d47-ad6d-3c1af2c0d1d9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:53.564095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bce773-5658-4d47-ad6d-3c1af2c0d1d9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:53.598705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:53.599324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.599973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.600248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.600890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:53.602371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:53.619990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.sql.core [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 02:28:53.619990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 02:28:53.619990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [parameters: {'user_id': '13e105257b3c4a36b127211de41102a8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 02:28:53.619990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80497) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application conn.query(q) Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result.read() Aug 09 02:28:53.626608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 02:28:53.627520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application next(self.gen) Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application yield resource Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application next(self.gen) Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise self.value Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application session.commit() Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 02:28:53.628504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.session.flush() Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application compat.raise_( Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 02:28:53.630028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 02:28:53.631182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application conn.query(q) Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application result.read() Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 02:28:53.632230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [parameters: {'user_id': '13e105257b3c4a36b127211de41102a8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:53.633278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 02:28:53.634388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:53.635539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 487/963] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 9 02:28:53 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:53.642214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:53.749708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550806-f71e-4a24-90ad-82c334e1124d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:53.787215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:53.787958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.788236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.788506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.796088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:53.797533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-9795f57f-16b4-44aa-bc16-80a4b23a19a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:53.809990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:53.810977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:53.811955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 477/964] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:28:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:53.817130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:53.869821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:53.926451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88883cd6-5f67-4be2-99ce-8f38c5834467 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:53.961683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:53.962552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:53.962938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:53.963308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:53.964118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:53.964524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:53.966309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017b36f-c6e0-4e7c-bcb6-b3ccedc49cbc None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:54.004934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 488/965] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 9 02:28:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:54.444178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b45a6-0586-44d1-a3c1-b2e31c7a28e6 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.444178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b45a6-0586-44d1-a3c1-b2e31c7a28e6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.444178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b45a6-0586-44d1-a3c1-b2e31c7a28e6 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.445267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 478/966] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:54.450625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e47c-b677-4ff0-8eff-7c3209f6d6c9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.450920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e47c-b677-4ff0-8eff-7c3209f6d6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.451197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e47c-b677-4ff0-8eff-7c3209f6d6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.469859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a661e47c-b677-4ff0-8eff-7c3209f6d6c9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:54.471409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a661e47c-b677-4ff0-8eff-7c3209f6d6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:54.540522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 489/967] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:54.562646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b45a6-0586-44d1-a3c1-b2e31c7a28e6 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:54.612488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:54.672435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b45a6-0586-44d1-a3c1-b2e31c7a28e6 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:54.700218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce31b6-ad43-4a10-8b01-c7377e18b046 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.700508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce31b6-ad43-4a10-8b01-c7377e18b046 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.700751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce31b6-ad43-4a10-8b01-c7377e18b046 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.705705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 490/968] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:54.706433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:54.707074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.707354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.707455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.707644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.707742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.707889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.708199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:54.709576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:54.718572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:54.719605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:54.723107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 479/969] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:54.726448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:54.727557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:54.728954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:54.756474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:54.801245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 491/970] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:54.823851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:54.827834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e594e48-e260-4830-8af4-ef7b41090e83 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:54.835406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:54.863573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:54.864189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.864467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.864854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.865485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:54.865788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:54.867171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:54.872156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 480/971] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:54.877681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:54.926406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0724-5d87-4d19-af04-cc964fa998f9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:54.962978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:54.963929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:54.964212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:54.964466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:54.974965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:28:54.979770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:54.980701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:28:54.981567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:54.984749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8d2c-7142-40c5-b381-8820513f98bc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:54.986046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 492/972] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 9 02:28:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:54.987088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.012131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.023824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.024450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.024759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.025027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.025660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:55.025958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.027355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.037166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 481/973] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:55.038738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.076357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.098913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb56a-9caf-497c-a5e3-c57ce055f8f3 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.135537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.136157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.136442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.136718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.137403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:55.137709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.139131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.146862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e51b0-3511-4483-a6fb-efa58f641da1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.147447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 493/974] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:28:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:55.153039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.184964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.185507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.185815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.186063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.186820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] RBAC: Adding query filter params (interface=admin, service_id=c8eceff23f24449fba22129c0e1d9543, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:55.187132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.188501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d13acd-7993-4949-9b0b-3b1d614612c5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.193480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 482/975] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/endpoints?service_id=c8eceff23f24449fba22129c0e1d9543&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:55.195711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aee7d-1fc1-4022-a411-063f9c75ca9f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.195998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aee7d-1fc1-4022-a411-063f9c75ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.196275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aee7d-1fc1-4022-a411-063f9c75ca9f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.197622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 483/976] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:55.199088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aee7d-1fc1-4022-a411-063f9c75ca9f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.241931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.257793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aeb917-64f4-4f07-a9fe-fe9685ef19c3 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.291452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.292097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.292351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.292628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.294824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.299825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aee7d-1fc1-4022-a411-063f9c75ca9f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.304127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.305581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-6a7f8cc4-55e8-4240-a0b4-22e8426176dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:55.320139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:55.321110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:55.322099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:55.322099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:55.322099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:55.322099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 494/977] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:55.322584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.322958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.323169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.334308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.334920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.335183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.335448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.336065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.337655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.342278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:55.343476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:55.343574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 484/978] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:28:55.344319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.443649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 495/979] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:55.461200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e76fc-8f0f-4fbd-84bc-481ac4832ed4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.466190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.497858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.498626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.498903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.499162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.499945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:55.500238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.501618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.516200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 485/980] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:55.522098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.545005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.548063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.580660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69b4d5-9b54-405d-881d-92e93e8be4dc None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.618650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.619276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.619562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.619832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.620496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.621884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.629402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254334e-2613-4305-8b3c-dddd34a59c93 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.637549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 496/981] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:28:55 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:28:55.674587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.675250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.675530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.675803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.684339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.685820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:28:55.699173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:55.700199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 02:28:55.701181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:55.705376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 486/982] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:55.706297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.816381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231903a1-7568-4305-8579-00b067ae5958 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:55.842021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.852062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:55.852815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:55.853168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:55.853508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:55.854350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:55.854731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:55.856248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:55.874023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 497/983] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:55.879951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:55.960876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:55.983519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbdaaa-4e98-4629-a6db-47a83220fb27 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:56.017215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:56.017829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:56.018106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:56.018413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:56.084575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:56.086053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:56.169842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:56.194793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 487/984] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:28:55 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 02:28:56.200978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:56.304333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694e0d6-b65a-4ae5-b362-a1543cebc1a7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:56.338067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:56.338751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:56.339029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:56.339296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:56.349946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:56.351423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:28:56.365657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:56.366949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:28:56.372804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:28:56.372804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:28:56.372804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:56.372804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 498/985] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:56.374878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:56.488888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aa3b9-a884-4473-a5e3-7beffc564c51 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:56.523653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:56.524394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:56.524799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:56.525185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:56.525928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:56.526362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:56.527820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:56.539240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 488/986] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:56.544968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:56.581854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:56.648211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a662b8-012a-4755-b836-97f7ede6045c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:56.680712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:56.681493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:56.681866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:56.682288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:56.688981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:56.690556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:28:56.701514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:56.702667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:28:56.703813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:56.705314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 499/987] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:56.710995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:56.751468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:56.814520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad8b29-c700-436b-b991-fd0a20a48388 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:56.848755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:56.849276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:56.849897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:56.850199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:56.850877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:56.851179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:56.852583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6fee3-9795-4b71-b2ba-3fa2606a3bf8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:56.867745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 489/988] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:56.870400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:56.968399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:56.975933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda31eb-0cf4-49c5-a7b0-1d9ee06b5e19 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:57.009512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:57.010234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.010586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.010936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.011733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] RBAC: Adding query filter params (role.id=9e89c752b6b94dcdbdc07bb30bc586db, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=562c6203e9cf440ebd81d14877c9e452) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:57.012117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:57.013711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6dd55-61ce-4743-b909-aa45f6918f10 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:57.019904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 500/989] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:28:56 2022] GET /identity/v3/role_assignments?user.id=562c6203e9cf440ebd81d14877c9e452&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=9e89c752b6b94dcdbdc07bb30bc586db => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:57.358783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47d1f-6bc2-4d06-9ba1-bdc85472b85e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.359218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47d1f-6bc2-4d06-9ba1-bdc85472b85e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.359338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47d1f-6bc2-4d06-9ba1-bdc85472b85e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.360704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 490/990] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:57.366035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.366547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.366627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.385399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:57.386619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:57.456397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 501/991] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:57.478944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47d1f-6bc2-4d06-9ba1-bdc85472b85e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:57.578074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:57.584896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47d1f-6bc2-4d06-9ba1-bdc85472b85e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:57.619757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:57.620324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.620603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.620880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.621458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:57.622857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e9b33073-9079-49bc-99a0-ab23a38ad830 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:57.631655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:57.632811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 02:28:57.634219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:57.635437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 491/992] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 9 02:28:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:57.638871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:57.650599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801041a-1ba8-4522-b989-2e7157ad359a None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.650869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801041a-1ba8-4522-b989-2e7157ad359a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.651143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801041a-1ba8-4522-b989-2e7157ad359a None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.652477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 492/993] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:28:57.657321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.657591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.657880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.676639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:57.677816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:57.730671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:57.743018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b179-380e-43dd-8147-369fb41f47b0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:57.746044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 493/994] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:28:57.767374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:57.777475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:57.778111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.778432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.778709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.779350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:57.779661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:57.781048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:57.790878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 502/995] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:57.791652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:57.868078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:57.878659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a43321-8626-4513-b417-de7a5749a292 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:57.901500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ede6-a1ff-43e4-9031-4ff62d28ffb4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:57.914845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:57.915531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.915810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.916373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.916658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:57.918018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:57.926699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:57.927718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:28:57.928608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:28:57.932948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 494/996] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:28:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:28:57.934013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:57.938255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:57.938971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:57.939317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:57.939656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:57.940366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:57.940738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:57.942198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:57.947368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 503/997] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 9 02:28:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:57.952731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:57.971922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:58.040471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1476e-1530-40ef-88e0-11d9db372257 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:58.053031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fe22c-5d09-4b4a-bd44-32d58af31e5b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:58.075285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:58.075903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:58.076151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:58.076428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:58.077043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:58.077328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:58.078719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:58.088282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 495/998] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:58.089511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:58.093859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:58.094522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:58.094801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:58.095075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:58.095747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:58.097124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:58.114056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:58.118026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 504/999] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:28:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:28:58.210484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72d1a4-baa0-4fc8-bd8f-0ffc6cf2e9e1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:58.255973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:58.256593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:58.256869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:58.257133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:58.257772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:58.258066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:58.259571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:58.265258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 496/1000] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:28:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:28:58.272048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:58.352444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:58.376767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dad20-6963-45af-acc5-b08e401f98a9 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:58.419952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:58.420580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:58.420860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:58.421135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:58.421851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] RBAC: Adding query filter params (interface=public, service_id=4ad06e432f1a49ea979d6734193ecd0f, region_id=RegionOne) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:58.422173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:58.423564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e2fdb-92e7-4291-bbea-3f3f8eb8fef0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:58.429046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 505/1001] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:28:58 2022] GET /identity/v3/endpoints?service_id=4ad06e432f1a49ea979d6734193ecd0f&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:58.434510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:58.522484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:58.543108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029241f-f045-4650-aa8b-34c988e7e595 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:58.578026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8737787-fdb5-4413-b88c-9923a3b13301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:58.578675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737787-fdb5-4413-b88c-9923a3b13301 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:58.579255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737787-fdb5-4413-b88c-9923a3b13301 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:58.579255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737787-fdb5-4413-b88c-9923a3b13301 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:58.579806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8737787-fdb5-4413-b88c-9923a3b13301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:58.581171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8737787-fdb5-4413-b88c-9923a3b13301 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:58.587908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 497/1002] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:28:58 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:28:59.048705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4186461-a247-4ad7-99e5-0dbfd38d4661 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:59.049067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4186461-a247-4ad7-99e5-0dbfd38d4661 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:59.049254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4186461-a247-4ad7-99e5-0dbfd38d4661 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:59.050908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 506/1003] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:28:59.055969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:59.056333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:59.056681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:59.075661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:28:59.076869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:28:59.147922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 498/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:28:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:28:59.169405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4186461-a247-4ad7-99e5-0dbfd38d4661 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:59.274609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4186461-a247-4ad7-99e5-0dbfd38d4661 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:59.308434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:59.308998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:59.309294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:59.309571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:59.310167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:59.311564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:28:59.320325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:28:59.321361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:28:59.322679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:28:59.326455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 507/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 9 02:28:59 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:28:59.327768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:59.402550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:59.433672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07470818-5400-41f3-a1a3-ad7f525b1a62 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:59.467885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:59.468502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:59.468780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:59.469047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:59.469683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] RBAC: Adding query filter params (name=image) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:59.469969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:59.471362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007dd96-1a5f-43a7-860b-94609715d055 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:59.476762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 499/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:28:59.482540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:28:59.496456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:28:59.592692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649626ca-86af-42a6-bea3-e7b9f3c7d3a8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:28:59.628230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:28:59.628905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:28:59.629186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:28:59.629458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:28:59.630092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] RBAC: Adding query filter params (type=image) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:28:59.630430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:28:59.631811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ecd7a-e34f-4b35-b785-0c296f964019 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:28:59.646185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 508/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:28:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:00.113037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d13d9-9354-4c97-b047-8f6096add013 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.113605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d13d9-9354-4c97-b047-8f6096add013 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.113605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d13d9-9354-4c97-b047-8f6096add013 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.115189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 500/1008] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:00.120430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.120721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.120999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.139080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:00.140287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:00.202655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:00.208499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 509/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:29:00.231779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d13d9-9354-4c97-b047-8f6096add013 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:00.342313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d13d9-9354-4c97-b047-8f6096add013 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:00.379430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722a9d4-e37e-4ccc-8199-11b05c359643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:00.380309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a9d4-e37e-4ccc-8199-11b05c359643 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.380740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a9d4-e37e-4ccc-8199-11b05c359643 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.381148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a9d4-e37e-4ccc-8199-11b05c359643 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.382474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a9d4-e37e-4ccc-8199-11b05c359643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:00.384100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a9d4-e37e-4ccc-8199-11b05c359643 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:00.385960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 501/1010] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:00.392021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:00.503325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07606285-2a22-464e-a8dd-3c569c3c9b2a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:00.538139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5c229-2a5c-41dc-9096-d5bb033d43cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:00.538781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5c229-2a5c-41dc-9096-d5bb033d43cd None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.539056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5c229-2a5c-41dc-9096-d5bb033d43cd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.539327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5c229-2a5c-41dc-9096-d5bb033d43cd None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.539978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5c229-2a5c-41dc-9096-d5bb033d43cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:00.541746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5c229-2a5c-41dc-9096-d5bb033d43cd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:00.574368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 510/1011] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:29:00 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:00.586142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d16da-d2ea-4f76-9f83-0d637da8a653 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.586468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d16da-d2ea-4f76-9f83-0d637da8a653 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.586742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d16da-d2ea-4f76-9f83-0d637da8a653 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.588299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 502/1012] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:00.594908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.595242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.595491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.615639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:00.616867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:00.670110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:00.696310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 511/1013] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:00.717461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d16da-d2ea-4f76-9f83-0d637da8a653 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:00.727866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:00.828792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d16da-d2ea-4f76-9f83-0d637da8a653 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:00.862401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:00.863034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:00.863306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:00.863569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:00.864147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:00.865491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:00.874272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:00.875282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:00.876206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 503/1014] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:29:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:00.881593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:00.989766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcc60e-a4b3-4bc7-a280-ec8e9e53d4e5 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:01.024261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:01.024908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:01.025189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:01.025457 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:01.026093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:01.026450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:01.027829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:01.037473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 512/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:29:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:01.038802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:01.110799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:01.143743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9416c7-b531-41b2-bdd4-3c6562b40a0a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:01.178590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:01.179199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:01.179477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:01.179744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:01.180374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:01.180667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:01.182067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3aea1-25e7-4c0b-b671-473d1054bbe4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:01.191821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 504/1016] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 9 02:29:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:01.192808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:01.241114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:01.297851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4eb2-3642-4f91-b773-84365d01ad45 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:01.334017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff46c2-adfe-4efd-b50c-8d2196f06c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:01.334669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff46c2-adfe-4efd-b50c-8d2196f06c3a None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:01.334950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff46c2-adfe-4efd-b50c-8d2196f06c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:01.335221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff46c2-adfe-4efd-b50c-8d2196f06c3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:01.335899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff46c2-adfe-4efd-b50c-8d2196f06c3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:01.337351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff46c2-adfe-4efd-b50c-8d2196f06c3a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:01.366198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 513/1017] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:29:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:01.632894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c5130-c37e-4224-ada2-29224bf64c3d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:01.633198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c5130-c37e-4224-ada2-29224bf64c3d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:01.633439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c5130-c37e-4224-ada2-29224bf64c3d None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:01.635092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 505/1018] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:01.640490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257f607-a915-4901-8dff-05900e52a6a5 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:01.640952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257f607-a915-4901-8dff-05900e52a6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:01.641056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257f607-a915-4901-8dff-05900e52a6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:01.660236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-9257f607-a915-4901-8dff-05900e52a6a5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:01.661519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9257f607-a915-4901-8dff-05900e52a6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:01.738928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 514/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:01 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:01.761681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c5130-c37e-4224-ada2-29224bf64c3d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:01.864959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c5130-c37e-4224-ada2-29224bf64c3d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:01.899555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d763c0-4acc-4403-9360-817db19d7f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:01.900122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d763c0-4acc-4403-9360-817db19d7f79 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:01.900404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d763c0-4acc-4403-9360-817db19d7f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:01.900680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d763c0-4acc-4403-9360-817db19d7f79 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:01.901334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d763c0-4acc-4403-9360-817db19d7f79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:01.902750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d763c0-4acc-4403-9360-817db19d7f79 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:01.906248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:01.920321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 506/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:29:01 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:02.638482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940a93e-5fab-4784-8c2d-f81fa5d15c2a None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:02.638850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940a93e-5fab-4784-8c2d-f81fa5d15c2a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:02.639019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940a93e-5fab-4784-8c2d-f81fa5d15c2a None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:02.640434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 515/1021] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:29:02.648088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:02.648571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:02.648571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:02.670865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:02.672655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:02.750787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 507/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:02.774604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940a93e-5fab-4784-8c2d-f81fa5d15c2a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:02.802925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:02.879073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940a93e-5fab-4784-8c2d-f81fa5d15c2a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:02.912591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfcd75-b8cd-45b0-8c2e-5eb8dfee02d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:02.913224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfcd75-b8cd-45b0-8c2e-5eb8dfee02d9 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:02.913501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfcd75-b8cd-45b0-8c2e-5eb8dfee02d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:02.913761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfcd75-b8cd-45b0-8c2e-5eb8dfee02d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:02.914892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfcd75-b8cd-45b0-8c2e-5eb8dfee02d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:02.916380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfcd75-b8cd-45b0-8c2e-5eb8dfee02d9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:02.918111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 516/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:02.924130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:03.030070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f3e84-4487-445e-833e-fc04ae211293 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:03.052142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:03.068750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43b97e-82c2-4814-95c8-b36fd211094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:03.069369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43b97e-82c2-4814-95c8-b36fd211094e None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.069647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43b97e-82c2-4814-95c8-b36fd211094e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.069919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43b97e-82c2-4814-95c8-b36fd211094e None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.070607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43b97e-82c2-4814-95c8-b36fd211094e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:03.072089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43b97e-82c2-4814-95c8-b36fd211094e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:03.095035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 508/1024] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:29:02 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:03.461637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3132c-19c1-4beb-a7f6-50cf0f8a331b None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.461887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3132c-19c1-4beb-a7f6-50cf0f8a331b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.462207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3132c-19c1-4beb-a7f6-50cf0f8a331b None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.463782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 517/1025] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:03.469019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.469287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.469575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.488265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:03.489470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:03.558589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 509/1026] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:03.581165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3132c-19c1-4beb-a7f6-50cf0f8a331b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:03.680089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:03.688746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3132c-19c1-4beb-a7f6-50cf0f8a331b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:03.723307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:03.723935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.724208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.724476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.734857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:03.739491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:03.740490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:29:03.741374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:03.745715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 518/1027] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 9 02:29:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:03.746796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:03.843918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0893657-28c4-4364-bfa7-613552790f9d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:03.851925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:03.880899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:03.881512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.881782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.882048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.882838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:03.883136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:03.884529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86831b72-0eaa-46a8-a10b-26847897188c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:03.892811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 510/1028] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:29:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:03.898586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:03.899202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf51f6-eaf2-40dc-8f8a-cb2114aee237 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.899496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf51f6-eaf2-40dc-8f8a-cb2114aee237 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.899900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf51f6-eaf2-40dc-8f8a-cb2114aee237 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.901181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 511/1029] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:03.906111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:03.906434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:03.906705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:03.925159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:03.926363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:04.002221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 512/1030] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:29:04.009641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d0b4-f98d-4d50-afd9-eb6eb54a5e3d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.031415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.043481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.044115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.044370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.044632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.055484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.057098 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:04.071640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:04.072625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:04.074567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:04.074567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:04.074567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:04.074567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 519/1031] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:04.078337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.089785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.130144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.136543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b421ccc-e988-41af-a46f-0eac222dbcf1 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.172109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.173097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.173596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.174066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] PATH_INFO: `/v3/services/image` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.174902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.176465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.184601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08c2e-67c7-4781-a372-c7dfc413175b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:04.186242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:04.187300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:29:04.188283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:04.195427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 513/1032] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:04.196500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.220247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.220856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.221139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.221397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.222036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:04.222370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.223751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.241452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 520/1033] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:04.242807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.314243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.315935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b5336-c837-4ff9-9b6f-2515231f3b47 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.349574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01ee0-7e74-49ec-8607-978f3ae04230 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.354430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.355049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.355326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.355598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.356227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] RBAC: Adding query filter params (name=image) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:04.356530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.357895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.363592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 514/1034] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:04.369066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.385793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.386459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.386752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.386993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.393947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.395403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.396184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:29:04.409933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:04.411003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:29:04.412535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:04.416304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 521/1035] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:04.417307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.478382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee14dc3-26f1-4d96-856e-1292f0777791 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.515300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.515924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.516205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.516482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.517141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] RBAC: Adding query filter params (type=image) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:04.517444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.518842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.520755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b11f76-3e1a-43a1-bbeb-39d187f543c1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.525583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 515/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:04.531415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.551320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.560882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.561614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.561944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.562245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.563013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:04.563318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.564911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.587892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 522/1037] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:04.589568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.621898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.632383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4f295-ba5d-4523-9221-7548bf91e6fa None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.667830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.668447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.668756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.669033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.669727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] RBAC: Adding query filter params (interface=public, service_id=b634ebfcb54442f0ac08af7a572f876c, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:04.670128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.671617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.677292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 516/1038] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/endpoints?service_id=b634ebfcb54442f0ac08af7a572f876c&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:04.682519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.695133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb537e-f433-4852-8cd5-ed8b05aac023 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.730298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.730926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.731208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.731481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.784523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:04.790987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae0ade-91e0-40c2-86f4-692cdd7b75fb None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:04.801629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.803189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.824263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:04.824811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:04.825084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:04.825356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:04.825963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:04.827369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:04.833899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 517/1039] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:29:04.909664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 523/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:04.915355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:04.937710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.026420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf4c3f-abe4-4ea7-851d-61af6d40a43e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.059356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.060012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.060276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.060548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.070923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.072423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3cd74dae-9c12-406b-81f9-26822db07e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:05.086199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:05.087265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:05.088197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:05.088197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:05.088197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:05.088197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 518/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:05.093604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.141837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9520-65dd-46a2-b772-64e0c26ed86e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.142115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9520-65dd-46a2-b772-64e0c26ed86e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.142435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9520-65dd-46a2-b772-64e0c26ed86e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.143927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 519/1042] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:29:05.148679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.148978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.149250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.168446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:05.169630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:05.185321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.202783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e3078-c2aa-4398-9899-9eb9397867b0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.234960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.241030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.241650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.241927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.242225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.242481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 520/1043] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:29:05.242882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:05.243182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.244654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.260966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 524/1044] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:05.262666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.263224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.365045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe5cf0-2cbe-4e7f-bfd2-7309cc6beb8f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.374511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ee076-b4c0-481b-8ac1-aa9c0da725b2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.398663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.399779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.400063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.400473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.411499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:05.416303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:05.417272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:05.418560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 525/1045] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:05.421553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.422229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.422812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.423091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.423380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.431079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.432251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.433091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:29:05.445790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:05.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:05.447756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 521/1046] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:05.452717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.516444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.529455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09939e-97be-4493-9133-b3c81e6da003 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.560011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e48b27-9e8b-4c43-b041-63df16a0daad None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.566866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.567976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.568263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.568546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.569572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:05.569884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.571798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.581608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 526/1047] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:05.587272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.596394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.596993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.597274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.597551 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.598280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:05.598591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.600393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.613019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 522/1048] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:05.619072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.658129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.698914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd7a2-7d9d-43a7-a72f-fc86ed1483c6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.722112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af9149-2e6f-4f34-83ec-df9feb681be6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.736306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.736959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.737224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.737498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] PATH_INFO: `/v3/users/9de0c57af93a44b99414d5f581a1d755` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.744111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.752883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] RBAC: Authorizing `identity:get_user(user_id=9de0c57af93a44b99414d5f581a1d755)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.754398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.760688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 527/1049] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/users/9de0c57af93a44b99414d5f581a1d755 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:05.760843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.760843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.760843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.760843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.761169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] RBAC: Adding query filter params (role.id=9e89c752b6b94dcdbdc07bb30bc586db, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=ad4a7ffff8f842969d782603cbb8fb89) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:05.761269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.762496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:05.762715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:05.769109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 523/1050] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/role_assignments?user.id=ad4a7ffff8f842969d782603cbb8fb89&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=9e89c752b6b94dcdbdc07bb30bc586db => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:05.878620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b3044-60cb-4f03-a75e-57a1a0b0d200 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:05.907916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:05.918476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:05.919166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:05.919380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:05.919660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:05.992702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:05.994407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:06.094666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 528/1051] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 02:29:06.100814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:06.200557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0a0e-59c2-42d0-9c17-c1f9d7afb407 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:06.236883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772c64c-9880-4b77-b3dc-f410225c98be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:06.237535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772c64c-9880-4b77-b3dc-f410225c98be None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:06.237817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772c64c-9880-4b77-b3dc-f410225c98be None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:06.238087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772c64c-9880-4b77-b3dc-f410225c98be None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:06.239244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772c64c-9880-4b77-b3dc-f410225c98be None admin] RBAC: Authorizing `identity:get_project(project_id=7f804c64c4b44ea5858d9185a34c8369)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:06.240689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772c64c-9880-4b77-b3dc-f410225c98be None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:06.242534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 524/1052] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:06 2022] GET /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:06.248877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:06.277301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:06.355612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115c28d-fab9-42a3-94f6-47befd49340e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:06.390526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:06.391140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:06.391421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:06.391695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:06.392431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=7f804c64c4b44ea5858d9185a34c8369, user.id=9de0c57af93a44b99414d5f581a1d755) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:06.392738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:06.394186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045f2d8-2c02-4206-a183-6f99061576e6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:06.407358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 529/1053] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:06 2022] GET /identity/v3/role_assignments?user.id=9de0c57af93a44b99414d5f581a1d755&scope.project.id=7f804c64c4b44ea5858d9185a34c8369&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:06.857155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c017f-a4cb-4092-ae48-e48c5c41ac0e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:06.857439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c017f-a4cb-4092-ae48-e48c5c41ac0e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:06.857726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c017f-a4cb-4092-ae48-e48c5c41ac0e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:06.859327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 525/1054] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:06.864930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:06.865213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:06.865513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:06.883907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:06.885158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:06.955681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 530/1055] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:06.978774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c017f-a4cb-4092-ae48-e48c5c41ac0e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:07.034533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:07.084216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c017f-a4cb-4092-ae48-e48c5c41ac0e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:07.118290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:07.118901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.119216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.119493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:07.120089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:07.121467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:07.130309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:07.131332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 02:29:07.132212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:07.134622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 526/1056] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 9 02:29:06 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:07.137769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:07.159027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:07.237387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c2c64-36d3-491b-943a-625d7c91bfce None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:07.270892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:07.271509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.271786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.272052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:07.272686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] RBAC: Adding query filter params (name=image) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:07.272993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:07.274392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:07.279465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 531/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:07.285179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:07.385717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d236b-e19e-4a5c-9fdb-01e01c7500e1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:07.401687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:07.425836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:07.426498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.426777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.427049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:07.427693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] RBAC: Adding query filter params (type=image) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:07.427997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:07.429369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7dac-d71d-468c-af04-f77e5cb2e4cd None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:07.434746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 527/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:07.440496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:07.527635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:07.544278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca30c9-87ce-4b9b-8dcf-993f4568d77c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:07.577929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc72c56-c375-49f3-9365-5d4d8f5dd16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:07.578603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc72c56-c375-49f3-9365-5d4d8f5dd16f None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.578881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc72c56-c375-49f3-9365-5d4d8f5dd16f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.579155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc72c56-c375-49f3-9365-5d4d8f5dd16f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:07.579832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc72c56-c375-49f3-9365-5d4d8f5dd16f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:07.581201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc72c56-c375-49f3-9365-5d4d8f5dd16f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:07.603830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 532/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:29:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:07.730805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377e124-30a6-4bce-86a7-4d47ae96a9db None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.730994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377e124-30a6-4bce-86a7-4d47ae96a9db None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.731243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377e124-30a6-4bce-86a7-4d47ae96a9db None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:07.732796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 528/1060] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:07.738345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565804-e2f3-434f-841b-081078383644 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.738655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565804-e2f3-434f-841b-081078383644 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.738897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565804-e2f3-434f-841b-081078383644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:07.759850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-55565804-e2f3-434f-841b-081078383644 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:07.761038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-55565804-e2f3-434f-841b-081078383644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:07.835688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 533/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:07.858449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377e124-30a6-4bce-86a7-4d47ae96a9db None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:07.963874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377e124-30a6-4bce-86a7-4d47ae96a9db None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:07.998493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:07.999048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:07.999328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:07.999594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.009735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:08.014524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:08.015459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:29:08.016337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:08.018230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 529/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 9 02:29:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:08.022032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565804-e2f3-434f-841b-081078383644 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.092286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.129570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565804-e2f3-434f-841b-081078383644 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.165420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.166031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.166334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.166598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.167259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:08.167542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.168905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.177422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 534/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:08.183403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.229066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.288636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c906d1-1dda-4817-9a68-eccee492c5b7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.322303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.322947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.323221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.323490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.333468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.334966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-81083626-5515-4594-9dea-f7afcbd775bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:08.349366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:08.350932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:08.351867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:08.351867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:08.351867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:08.351867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 530/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:08.358445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.358681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa6145-0d29-4bb7-9dba-7f1ed27059dd None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.358969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa6145-0d29-4bb7-9dba-7f1ed27059dd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.359242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa6145-0d29-4bb7-9dba-7f1ed27059dd None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.360786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 531/1065] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:29:08.365531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.365813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.366141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.385088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:08.386305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:08.460843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 532/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:29:08.470911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7cbf-119f-492e-89a5-462c2190146d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.483447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.497478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.506461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.507098 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.507373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.507637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.508283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:08.508582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.508898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.510026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.523075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 535/1067] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:08.528295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.589504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29956db-5ef8-4daa-9c7c-f65d18441587 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.627856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.628517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.628788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.629064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.630619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dca378-9bb8-4240-a014-a3e11cac92bd None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.639459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:08.644158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:08.645190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:08.646071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 533/1068] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:08.651272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.667406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.668032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.668307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.668562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.675859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.677307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:29:08.693095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:08.694347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:29:08.695471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:08.700996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 536/1069] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:08.701781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.749797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.759634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.770252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b30b4c-902f-4da6-8ea8-893f6780609d None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.807893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.808530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.808806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.809071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.809730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:08.810033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.811435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.812981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a01c6c-af60-4c61-be99-3fac3327d8ec None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.819789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 534/1070] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:08.825510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.847156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.847840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.848216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.848586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.849413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:08.849817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.857732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.877319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 537/1071] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:08.882910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.927000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbda3e-270c-44bf-8438-2e023e2322db None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.961072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:08.961808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:08.962116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:08.962576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] PATH_INFO: `/v3/users/9de0c57af93a44b99414d5f581a1d755` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:08.963672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] RBAC: Authorizing `identity:get_user(user_id=9de0c57af93a44b99414d5f581a1d755)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:08.965227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:08.967226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 535/1072] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/users/9de0c57af93a44b99414d5f581a1d755 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:08.972588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:08.986365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10312f8a-6cc8-4714-a576-eb32fa09213b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:08.988007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:08.994983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:09.023068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.023706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.023963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.024231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.034092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.035566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:09.050819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:09.051913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:09.052862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:09.052862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:09.052862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:09.057300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 538/1073] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:09.057612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:09.081222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd75f8-3faa-471f-8352-ee32a757233e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.116662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.117541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.117950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.118372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.119709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] RBAC: Authorizing `identity:get_project(project_id=7f804c64c4b44ea5858d9185a34c8369)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.121702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.128461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 536/1074] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:08 2022] GET /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:09.129417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:09.157837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8bd0-5bb5-4084-a0a6-8d7db58aa801 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.194362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.194978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.195273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.195531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.196200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:09.196477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.197886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.210662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 539/1075] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:09.216364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:09.221643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:09.238426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:09.241489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad4119-d3d9-4473-9a8d-77342d4f73c0 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.277342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.278014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.278329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.278596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369/users/9de0c57af93a44b99414d5f581a1d755/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.281155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f804c64c4b44ea5858d9185a34c8369, user_id=9de0c57af93a44b99414d5f581a1d755, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.282630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.299473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 537/1076] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:29:09 2022] PUT /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369/users/9de0c57af93a44b99414d5f581a1d755/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:29:09.328229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee41ae9-4bcc-48c3-8c2f-827bf2e769a4 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.366491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.367200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.367433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.367694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.375638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.377124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:29:09.389524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:09.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:09.396932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 540/1077] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:09.399306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:09.511196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182412dc-11c9-4daa-afed-2515c0f1c1a2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.546065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.546717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.546988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.547256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.547941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:09.548240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.549665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87437316-bf8e-4fa8-828b-2fffeaf217a2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.558841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:09.563878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 538/1078] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:09.569804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:09.614617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043afa-914e-425c-a02d-e408cd5091f2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.614911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043afa-914e-425c-a02d-e408cd5091f2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.615185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64043afa-914e-425c-a02d-e408cd5091f2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.616520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 539/1079] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:09.621675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.621913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.622193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.640903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:09.642088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:09.653861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:09.677787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75e19e-4606-4d37-93c8-cde9ddcf341e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.712924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.713896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.714197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.714479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/ad4a7ffff8f842969d782603cbb8fb89/roles/9e89c752b6b94dcdbdc07bb30bc586db` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.716731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=ad4a7ffff8f842969d782603cbb8fb89, role_id=9e89c752b6b94dcdbdc07bb30bc586db)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.718200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:09.720444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 540/1080] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:09.733964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 541/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:29:09 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/ad4a7ffff8f842969d782603cbb8fb89/roles/9e89c752b6b94dcdbdc07bb30bc586db => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:29:09.737696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:09.849908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2459e9c-0ec3-41a0-8577-954f40fdbc3c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:09.882352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:09.890846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:09.891474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:09.891754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:09.892026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:09.964017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:09.965769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:10.069153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 541/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:10.075624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:10.177679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a24e7f-006e-4670-ab86-a425df81859d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:10.212146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:10.212820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:10.213099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:10.213371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:10.220048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:10.226013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:10.227539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-4cf60202-23c2-4658-aa4b-f0e4e5d65825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:10.242121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:10.243120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:10.249353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:10.249353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:10.249353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:10.249353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 542/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:10.251051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:10.282826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:10.356490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efea64-c759-4071-acfe-1d852ac89230 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:10.390589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:10.391216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:10.391491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:10.391758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:10.392411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:10.392712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:10.394127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b095ad-3967-47df-89c3-213a6e6b587c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:10.407614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 542/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:11.286291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26a0bf-ceea-454a-8e36-c39f1121e2b0 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.286859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26a0bf-ceea-454a-8e36-c39f1121e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.286859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26a0bf-ceea-454a-8e36-c39f1121e2b0 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.288212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 543/1085] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:11.293521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.294048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.294048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.312934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:11.314110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:11.384581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 543/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:11.407539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26a0bf-ceea-454a-8e36-c39f1121e2b0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:11.511981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26a0bf-ceea-454a-8e36-c39f1121e2b0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:11.546347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:11.546893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.547171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.547433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.558047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:11.562669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:11.563656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:11.564528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 544/1087] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:11.569984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:11.592050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:11.670678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031b99-bf88-4ae0-9659-d74a33189a40 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:11.705674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:11.706308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.706553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.706849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.707504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:11.707804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:11.709172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ba49d-e5c3-495b-a3ea-1a6dabef7ec1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:11.717686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 544/1088] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:11.723488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:11.750498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fbcb-2e8d-4355-af1b-598498caf8aa None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.750779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fbcb-2e8d-4355-af1b-598498caf8aa None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.751027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fbcb-2e8d-4355-af1b-598498caf8aa None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.752548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 545/1089] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:11.757088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.757393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.757665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.775224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:11.775678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:11.776931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:11.825247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62567dfb-1ea6-4ffc-9db5-ed182e95f6f6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:11.841908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 546/1090] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:29:11.858037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:11.858704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:11.858983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:11.859236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] PATH_INFO: `/v3/users/9de0c57af93a44b99414d5f581a1d755` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:11.860384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] RBAC: Authorizing `identity:get_user(user_id=9de0c57af93a44b99414d5f581a1d755)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:11.861810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:11.863433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 545/1091] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:11 2022] GET /identity/v3/users/9de0c57af93a44b99414d5f581a1d755 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:11.865475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:11.869020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:11.879689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:11.963144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c07cbc-e8e2-4059-9b8e-6c3bb5611dc8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:11.968113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d269-9917-4cd2-9c70-8e5d225090d1 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:11.985765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.000171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.000810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.001090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.001360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.003400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.004391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.004673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.004941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.011473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:12.016160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:12.017173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:12.018410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 547/1092] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 9 02:29:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:12.023895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.071176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.072697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.102421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.131537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6200cb-d321-4f61-97df-b61dc3ff9dd9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.164798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.165431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.165684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.165952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.166657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:12.166963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.168548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.172680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 546/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 02:29:12.176270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 548/1094] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:12.178673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.181710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.196602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.275218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee84ce-c207-4280-a397-d0894f465992 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.279693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedd77a-2651-4fe2-9c61-08da5f12ab11 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.308770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.309394 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.309674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.309939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.311069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] RBAC: Authorizing `identity:get_project(project_id=7f804c64c4b44ea5858d9185a34c8369)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.311273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.311893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.312166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.312461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.312575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.314302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.314479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 547/1095] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:12.320509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.323459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.324893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:12.338621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:12.339589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:12.340519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:12.340519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:12.340519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:12.341049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 549/1096] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:12.346371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.407277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.426015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc4c13-9da2-4dce-a969-0ba370afdf35 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.449490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d467b-7893-4054-8515-4894d2b80f41 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.461322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.461932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.462254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.462525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.463256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=7f804c64c4b44ea5858d9185a34c8369, user.id=9de0c57af93a44b99414d5f581a1d755) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:12.463564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.465056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.470944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 548/1097] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/role_assignments?user.id=9de0c57af93a44b99414d5f581a1d755&scope.project.id=7f804c64c4b44ea5858d9185a34c8369&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:12.485391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.486021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.486335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.486603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.487274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:12.487576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.488995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.502652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 550/1098] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:12.508686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.615497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70e0e-aeec-4866-820f-f222f05abcc2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.651733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.652372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.652645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.652987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.660532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.661889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:29:12.672744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:12.673725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:12.674780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 549/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:12.679950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.725741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.778908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca6ca-4667-40f5-8cb8-990c92cbf400 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.813262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.813871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.814148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.814447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:12.815176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:12.815480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:12.816916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:12.834040 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 551/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:12.840062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:12.853889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:12.943856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c37a-657a-42e5-afcc-5c06b86acd8a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:12.980882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:12.981487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:12.981761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:12.982031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:13.049357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:13.050838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:13.061607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:13.149923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 550/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:13.155951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:13.256198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b635cd1-4fe3-47c9-8161-832c9eef1a70 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:13.258706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:13.291882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:13.292501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:13.292775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:13.293046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:13.303059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:13.304498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:13.318783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:13.320187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:29:13.321118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:13.321118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:29:13.321118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:13.321118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 552/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:29:13.325944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:13.432250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdcd2b-a196-463b-915b-e87cc6d50125 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:13.443088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:13.475961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:13.476577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:13.476830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:13.477112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:13.477755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:13.478023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:13.479470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:13.491364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 551/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:13.497637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:13.599159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cddd34-ac5a-4bd0-8f0c-fd9e5107543c None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:13.633760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:13.634467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:13.634747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:13.634994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:13.641797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:13.643275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:29:13.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:13.655550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:13.656921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 553/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:13.661800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:13.763865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a8ead-495e-4ebe-aba8-9bb18ada179a None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:13.797401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:13.798000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:13.798301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:13.798575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:13.799268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:13.799572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:13.801007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd8fb-816a-40de-a898-9df297fed6c0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:13.812895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 552/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:13.819123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:13.839382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:13.922687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45cf3d-f130-42c6-a5d2-8e8c61f0a3dc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:13.956666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:13.957270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:13.957546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:13.957812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:13.958575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] RBAC: Adding query filter params (role.id=9e89c752b6b94dcdbdc07bb30bc586db, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=ad4a7ffff8f842969d782603cbb8fb89) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:13.958882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:13.960309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd9868-69b7-4df9-9329-ce711f8dc83c None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:13.966170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 554/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:13 2022] GET /identity/v3/role_assignments?user.id=ad4a7ffff8f842969d782603cbb8fb89&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=9e89c752b6b94dcdbdc07bb30bc586db => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:14.466631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a110a-9297-4426-9e49-2f0eea945e3a None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:14.467162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a110a-9297-4426-9e49-2f0eea945e3a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:14.467162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a110a-9297-4426-9e49-2f0eea945e3a None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:14.468548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 553/1107] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:14.474244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:14.474690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:14.474778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:14.495198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:14.496428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:14.562383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 555/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:14.586519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a110a-9297-4426-9e49-2f0eea945e3a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:14.618818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:14.691037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a110a-9297-4426-9e49-2f0eea945e3a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:14.728798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79941f69-9906-454a-b128-a68e96c6d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:14.729443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79941f69-9906-454a-b128-a68e96c6d911 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:14.729723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79941f69-9906-454a-b128-a68e96c6d911 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:14.729992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79941f69-9906-454a-b128-a68e96c6d911 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:14.731170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79941f69-9906-454a-b128-a68e96c6d911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:14.732602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79941f69-9906-454a-b128-a68e96c6d911 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:14.734228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 554/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:14.740288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:14.752407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:14.839173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3694c-21ae-43e7-8345-8d7afdceb2d7 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:14.872804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff13ca-aeb9-4751-aa9f-f4064cc13ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:14.873417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff13ca-aeb9-4751-aa9f-f4064cc13ba7 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:14.873695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff13ca-aeb9-4751-aa9f-f4064cc13ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:14.873966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff13ca-aeb9-4751-aa9f-f4064cc13ba7 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:14.874643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff13ca-aeb9-4751-aa9f-f4064cc13ba7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:14.876085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff13ca-aeb9-4751-aa9f-f4064cc13ba7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:14.901924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 556/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:29:14 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:16.949631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d3138-cdd7-448b-bf68-0c8d881bcc3e None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:16.950564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d3138-cdd7-448b-bf68-0c8d881bcc3e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:16.950564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d3138-cdd7-448b-bf68-0c8d881bcc3e None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:16.952713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 555/1111] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:16.959068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:16.959797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:16.959935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:16.985046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:16.986258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:17.047698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:17.059181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 557/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:17.082579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d3138-cdd7-448b-bf68-0c8d881bcc3e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:17.193668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:17.199379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d3138-cdd7-448b-bf68-0c8d881bcc3e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:17.237452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:17.238017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:17.238348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:17.238619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] PATH_INFO: `/v3/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:17.240030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] RBAC: Authorizing `identity:get_role(role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:17.241401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:17.242751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 556/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:17 2022] GET /identity/v3/roles/14f7282e89144655a81a327a65c198a5 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:17.249920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:17.367284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd93e5-78b7-4da8-b5b1-a4e3d905a96e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:17.401341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:17.402031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:17.402342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:17.402616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] PATH_INFO: `/v3/users/7f0ff2c77a1a48529037e7f2fcd30629` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:17.416356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] RBAC: Authorizing `identity:get_user(user_id=7f0ff2c77a1a48529037e7f2fcd30629)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:17.417866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:17.419548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 558/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:17 2022] GET /identity/v3/users/7f0ff2c77a1a48529037e7f2fcd30629 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:17.425596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:17.538566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf522c-e021-4969-9adc-67b080cabc1f None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:17.573045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:17.573669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:17.573951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:17.574248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:17.609301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:17.654129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:17.655618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:17.754729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 557/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:17.765339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:17.823557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:17.868588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3a4e3-f88e-4476-adf2-3caf337efc13 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:17.904799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c4871-be6d-45ac-82e4-c665910bb533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:17.905428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c4871-be6d-45ac-82e4-c665910bb533 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:17.905709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c4871-be6d-45ac-82e4-c665910bb533 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:17.905988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c4871-be6d-45ac-82e4-c665910bb533 None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:17.907253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c4871-be6d-45ac-82e4-c665910bb533 None admin] RBAC: Authorizing `identity:get_project(project_id=7f804c64c4b44ea5858d9185a34c8369)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:17.908679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c4871-be6d-45ac-82e4-c665910bb533 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:17.910310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 559/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:17 2022] GET /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:17.919563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:17.997465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:18.043998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f1cb-a0da-46c7-a5c6-2df1eed4e023 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:18.079492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:18.080092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:18.080371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:18.080640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:18.081370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] RBAC: Adding query filter params (role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=7f804c64c4b44ea5858d9185a34c8369, user.id=7f0ff2c77a1a48529037e7f2fcd30629) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:18.081684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:18.083161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58f0d7-1c62-4412-b6a5-bccca031a8d6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:18.088598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 558/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:17 2022] GET /identity/v3/role_assignments?user.id=7f0ff2c77a1a48529037e7f2fcd30629&scope.project.id=7f804c64c4b44ea5858d9185a34c8369&role.id=14f7282e89144655a81a327a65c198a5 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:20.039322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543bfd-9982-4378-84ca-cc73e192d1bf None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:20.040249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543bfd-9982-4378-84ca-cc73e192d1bf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:20.040249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543bfd-9982-4378-84ca-cc73e192d1bf None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:20.042707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 560/1118] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:20.048712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:20.049387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:20.049504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:20.072948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:20.074186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:20.145424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 559/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:20.170568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d543bfd-9982-4378-84ca-cc73e192d1bf None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:20.279968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d543bfd-9982-4378-84ca-cc73e192d1bf None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:20.302665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:20.315441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:20.316056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:20.316329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:20.316599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] PATH_INFO: `/v3/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:20.318085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] RBAC: Authorizing `identity:get_role(role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:20.319472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:20.320766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 561/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:20 2022] GET /identity/v3/roles/14f7282e89144655a81a327a65c198a5 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:20.327479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:20.402370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:20.444337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac777e-a0fd-4095-af50-1893eb86733a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:20.483765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:20.484570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:20.485009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:20.485482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] PATH_INFO: `/v3/users/7f0ff2c77a1a48529037e7f2fcd30629` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:20.487281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] RBAC: Authorizing `identity:get_user(user_id=7f0ff2c77a1a48529037e7f2fcd30629)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:20.489256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:20.491132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 560/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:20 2022] GET /identity/v3/users/7f0ff2c77a1a48529037e7f2fcd30629 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:20.500061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:20.603048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1277c-39d6-4302-8cc8-604f5e3cf987 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:20.643852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909aa62-8b84-45e6-bcc5-19e917c30898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:20.644478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909aa62-8b84-45e6-bcc5-19e917c30898 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:20.644754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909aa62-8b84-45e6-bcc5-19e917c30898 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:20.645027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909aa62-8b84-45e6-bcc5-19e917c30898 None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:20.646141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909aa62-8b84-45e6-bcc5-19e917c30898 None admin] RBAC: Authorizing `identity:get_project(project_id=7f804c64c4b44ea5858d9185a34c8369)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:20.647608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909aa62-8b84-45e6-bcc5-19e917c30898 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:20.649231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 562/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:20 2022] GET /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:20.655835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:20.754615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfd89c-3503-4354-9022-ef5eb0d58b9e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:20.781291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:20.792865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b457d76-860f-4e42-9067-113979e9537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:20.793514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b457d76-860f-4e42-9067-113979e9537b None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:20.793792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b457d76-860f-4e42-9067-113979e9537b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:20.794064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b457d76-860f-4e42-9067-113979e9537b None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369/users/7f0ff2c77a1a48529037e7f2fcd30629/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:20.796384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b457d76-860f-4e42-9067-113979e9537b None admin] RBAC: Authorizing `identity:create_grant(project_id=7f804c64c4b44ea5858d9185a34c8369, user_id=7f0ff2c77a1a48529037e7f2fcd30629, role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:20.797817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b457d76-860f-4e42-9067-113979e9537b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:20.814775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 561/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:29:20 2022] PUT /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369/users/7f0ff2c77a1a48529037e7f2fcd30629/roles/14f7282e89144655a81a327a65c198a5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:29:22.766743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837854b-22ba-4239-825e-7ea984e5cf65 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:22.767301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837854b-22ba-4239-825e-7ea984e5cf65 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:22.767301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837854b-22ba-4239-825e-7ea984e5cf65 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:22.768554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 563/1124] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:22.774017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:22.774471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:22.774582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:22.794287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:22.795473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:22.863277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 562/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:22.886613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837854b-22ba-4239-825e-7ea984e5cf65 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:22.950973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:22.994948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837854b-22ba-4239-825e-7ea984e5cf65 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:23.029157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:23.029708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:23.029986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:23.030286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] PATH_INFO: `/v3/roles/14f7282e89144655a81a327a65c198a5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:23.031693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] RBAC: Authorizing `identity:get_role(role_id=14f7282e89144655a81a327a65c198a5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:23.033069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:23.034385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 564/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:22 2022] GET /identity/v3/roles/14f7282e89144655a81a327a65c198a5 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:23.041129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:23.154515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194c949-3bf9-4e1c-800a-c7fc1311f059 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:23.171626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:23.189288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:23.189908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:23.190213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:23.190485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] PATH_INFO: `/v3/users/7f0ff2c77a1a48529037e7f2fcd30629` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:23.191603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] RBAC: Authorizing `identity:get_user(user_id=7f0ff2c77a1a48529037e7f2fcd30629)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:23.193056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:23.194649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 563/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:23 2022] GET /identity/v3/users/7f0ff2c77a1a48529037e7f2fcd30629 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:23.200589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:23.309648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f36458-6a78-445c-b496-d58fb8437b52 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:23.335081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:23.345224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:23.345821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:23.346096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:23.346393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:23.414852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:23.417206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:23.519726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 565/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:23.526737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:23.643810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276ee7-6671-4052-964d-3217db4338e8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:23.681239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5991c-626b-4511-90ed-5b306cf326de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:23.681853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5991c-626b-4511-90ed-5b306cf326de None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:23.682132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5991c-626b-4511-90ed-5b306cf326de None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:23.682429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5991c-626b-4511-90ed-5b306cf326de None admin] PATH_INFO: `/v3/projects/7f804c64c4b44ea5858d9185a34c8369` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:23.683711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5991c-626b-4511-90ed-5b306cf326de None admin] RBAC: Authorizing `identity:get_project(project_id=7f804c64c4b44ea5858d9185a34c8369)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:23.685148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5991c-626b-4511-90ed-5b306cf326de None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:23.686802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 564/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:23 2022] GET /identity/v3/projects/7f804c64c4b44ea5858d9185a34c8369 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:23.693092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:23.720309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:23.811375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6fd3b-04c4-487d-91e9-ddda947ef684 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:23.849268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:23.849859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:23.850136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:23.850775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:23.851521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] RBAC: Adding query filter params (role.id=14f7282e89144655a81a327a65c198a5, scope.project.id=7f804c64c4b44ea5858d9185a34c8369, user.id=7f0ff2c77a1a48529037e7f2fcd30629) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:23.851833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:23.853233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4518ea6-214a-459f-b3cd-c46f6a5f265f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:23.859267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 566/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:23 2022] GET /identity/v3/role_assignments?user.id=7f0ff2c77a1a48529037e7f2fcd30629&scope.project.id=7f804c64c4b44ea5858d9185a34c8369&role.id=14f7282e89144655a81a327a65c198a5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:25.853389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9611-03fe-4509-b7cb-21d330301020 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:25.854269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9611-03fe-4509-b7cb-21d330301020 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:25.854269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9611-03fe-4509-b7cb-21d330301020 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:25.856370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 565/1131] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:25.862046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:25.862540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:25.862632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:25.880857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:25.882032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:25.952967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 567/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 02:29:25.988146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e9611-03fe-4509-b7cb-21d330301020 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:26.057013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:26.102874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e9611-03fe-4509-b7cb-21d330301020 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:26.139915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99434dc-cf24-41b8-9a71-ab99dbb9409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:26.140487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99434dc-cf24-41b8-9a71-ab99dbb9409d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:26.140766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99434dc-cf24-41b8-9a71-ab99dbb9409d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:26.141033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99434dc-cf24-41b8-9a71-ab99dbb9409d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:26.142191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99434dc-cf24-41b8-9a71-ab99dbb9409d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:26.143633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99434dc-cf24-41b8-9a71-ab99dbb9409d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:26.145244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 566/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:26.151295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:26.172693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:26.250077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7369887-03dc-4676-953c-a8fdf5ff2244 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:26.283937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d18e65-42e3-46d2-bbe4-d76be4c69111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:26.284539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18e65-42e3-46d2-bbe4-d76be4c69111 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:26.284813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18e65-42e3-46d2-bbe4-d76be4c69111 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:26.285081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18e65-42e3-46d2-bbe4-d76be4c69111 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:26.285724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18e65-42e3-46d2-bbe4-d76be4c69111 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:26.287184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18e65-42e3-46d2-bbe4-d76be4c69111 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:26.315963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 568/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:29:26 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:28.309121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d19c8-5f85-4946-8aed-97a1a96f59bd None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:28.310075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d19c8-5f85-4946-8aed-97a1a96f59bd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:28.310075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d19c8-5f85-4946-8aed-97a1a96f59bd None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:28.312243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 567/1135] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:28.320870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:28.321325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:28.321449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:28.347762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:28.348961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:28.419357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 569/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:28.444300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d19c8-5f85-4946-8aed-97a1a96f59bd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:28.558075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d19c8-5f85-4946-8aed-97a1a96f59bd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:28.581987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:28.595105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8422477-25a0-4b98-bda1-0cda09bfa31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:28.595734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422477-25a0-4b98-bda1-0cda09bfa31e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:28.596012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422477-25a0-4b98-bda1-0cda09bfa31e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:28.596284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8422477-25a0-4b98-bda1-0cda09bfa31e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:28.597452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8422477-25a0-4b98-bda1-0cda09bfa31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:28.598938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8422477-25a0-4b98-bda1-0cda09bfa31e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:28.600539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 568/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:29:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:28.607073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:28.609309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:28.714903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a997e5-84b6-48e6-bc18-44e9a7c8ef4a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:28.751491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3090e5-57c1-48ad-8dad-b69f0b19bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:28.752086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3090e5-57c1-48ad-8dad-b69f0b19bd34 None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:28.752370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3090e5-57c1-48ad-8dad-b69f0b19bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:28.752640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3090e5-57c1-48ad-8dad-b69f0b19bd34 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:28.753279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3090e5-57c1-48ad-8dad-b69f0b19bd34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:28.754750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3090e5-57c1-48ad-8dad-b69f0b19bd34 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:28.781703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:29:28 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:30.794737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8447a-848b-4f3f-a945-c55debf80594 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:30.795587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8447a-848b-4f3f-a945-c55debf80594 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:30.795587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8447a-848b-4f3f-a945-c55debf80594 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:30.797737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 569/1139] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:30.803130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:30.803570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:30.803669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:30.822299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:30.823474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:30.880243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:30.894981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 571/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:30.917940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8447a-848b-4f3f-a945-c55debf80594 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:31.030944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8447a-848b-4f3f-a945-c55debf80594 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:31.065784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:31.066385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:31.066663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:31.066935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:31.078685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:31.083409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:31.087561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:31.088791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 570/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:31.090899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:31.193463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0362278-5417-42ad-a31c-82c49cea7125 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:31.228256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:31.228856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:31.229134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:31.229406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:31.230073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:31.230388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:31.231862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:31.239755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 572/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:31.245760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:31.300017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:31.353146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385b50-0906-4554-bab2-36cd5ce34473 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:31.388646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:31.389256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:31.389527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:31.389794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] PATH_INFO: `/v3/users/676fb1f48bd549778d8ab97b3acd712a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:31.403421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] RBAC: Authorizing `identity:get_user(user_id=676fb1f48bd549778d8ab97b3acd712a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:31.404864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:31.406487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 571/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:31 2022] GET /identity/v3/users/676fb1f48bd549778d8ab97b3acd712a => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:31.413132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:31.473279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:31.537677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250ea16-6c23-4583-91e2-a753bedf3bab None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:31.573117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:31.573710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:31.573986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:31.574291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:31.654584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:31.656652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:31.709543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:31.760578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 573/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:31.766863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:31.878269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbc22f-b620-4e28-aa3f-9a91466030b9 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:31.887375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:31.926269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135001f-270b-42b7-9f00-a2aaeb3dbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:31.926852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135001f-270b-42b7-9f00-a2aaeb3dbaff None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:31.927130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135001f-270b-42b7-9f00-a2aaeb3dbaff None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:31.927403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135001f-270b-42b7-9f00-a2aaeb3dbaff None admin] PATH_INFO: `/v3/projects/3289e11e660c40aeb3bf4d37313032a8` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:31.928551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135001f-270b-42b7-9f00-a2aaeb3dbaff None admin] RBAC: Authorizing `identity:get_project(project_id=3289e11e660c40aeb3bf4d37313032a8)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:31.929995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135001f-270b-42b7-9f00-a2aaeb3dbaff None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:31.931680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 572/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:31 2022] GET /identity/v3/projects/3289e11e660c40aeb3bf4d37313032a8 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:31.941002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:32.048938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d59c2-77b5-468f-8d39-120cbeb85a75 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:32.083136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:32.083742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:32.084024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:32.084294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:32.085025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=3289e11e660c40aeb3bf4d37313032a8, user.id=676fb1f48bd549778d8ab97b3acd712a) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:32.085333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:32.086814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac12131-369c-42ae-94b2-20c95970dc9e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:32.092359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 574/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:31 2022] GET /identity/v3/role_assignments?user.id=676fb1f48bd549778d8ab97b3acd712a&scope.project.id=3289e11e660c40aeb3bf4d37313032a8&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:34.046210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb61f-9e3f-427d-ac13-6dd14918b180 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.047145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb61f-9e3f-427d-ac13-6dd14918b180 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.047145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb61f-9e3f-427d-ac13-6dd14918b180 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.049319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 573/1147] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:34.055543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.055980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.056115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.081806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:34.083029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:34.092611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:34.157518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 575/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:34.187891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cb61f-9e3f-427d-ac13-6dd14918b180 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:34.296509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cb61f-9e3f-427d-ac13-6dd14918b180 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:34.331438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:34.332053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.332316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.332590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.342716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:34.347412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:34.348382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:34.349275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 574/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:34.357159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:34.455103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75d1ba-19c2-4a7d-bd4a-4a43059a2730 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:34.490171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:34.490802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.491083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.491356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.492036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:34.492343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:34.493713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:34.501912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 576/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:34.508297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:34.544364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:34.617106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbfb07-3071-4ba2-b612-26f4dc55d207 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:34.651572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:34.652222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.652506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.652801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] PATH_INFO: `/v3/users/676fb1f48bd549778d8ab97b3acd712a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.653978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] RBAC: Authorizing `identity:get_user(user_id=676fb1f48bd549778d8ab97b3acd712a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:34.655447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:34.657034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 575/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:34 2022] GET /identity/v3/users/676fb1f48bd549778d8ab97b3acd712a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:34.666455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:34.677176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:34.791229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b796cf-9356-4240-bb69-a194e0bdf7ae None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:34.826632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f477d-90be-4279-9f07-3bd13d344f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:34.827253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f477d-90be-4279-9f07-3bd13d344f80 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.827534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f477d-90be-4279-9f07-3bd13d344f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.827802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f477d-90be-4279-9f07-3bd13d344f80 None admin] PATH_INFO: `/v3/projects/3289e11e660c40aeb3bf4d37313032a8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.829012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f477d-90be-4279-9f07-3bd13d344f80 None admin] RBAC: Authorizing `identity:get_project(project_id=3289e11e660c40aeb3bf4d37313032a8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:34.830470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f477d-90be-4279-9f07-3bd13d344f80 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:34.832119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 577/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:34 2022] GET /identity/v3/projects/3289e11e660c40aeb3bf4d37313032a8 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:34.840961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:34.955579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f34bf-c584-4a6f-af68-aca4ea8a76f6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:34.957825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:34.990607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7066d-44ea-4d1d-8735-e22bae0bf741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:34.991244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7066d-44ea-4d1d-8735-e22bae0bf741 None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:34.991527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7066d-44ea-4d1d-8735-e22bae0bf741 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:34.991797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7066d-44ea-4d1d-8735-e22bae0bf741 None admin] PATH_INFO: `/v3/projects/3289e11e660c40aeb3bf4d37313032a8/users/676fb1f48bd549778d8ab97b3acd712a/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:34.993774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7066d-44ea-4d1d-8735-e22bae0bf741 None admin] RBAC: Authorizing `identity:create_grant(project_id=3289e11e660c40aeb3bf4d37313032a8, user_id=676fb1f48bd549778d8ab97b3acd712a, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:34.995237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7066d-44ea-4d1d-8735-e22bae0bf741 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:35.011546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 576/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:29:34 2022] PUT /identity/v3/projects/3289e11e660c40aeb3bf4d37313032a8/users/676fb1f48bd549778d8ab97b3acd712a/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:29:36.961762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f47ac-5c2f-4704-bb4c-6203384cb7df None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:36.962679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f47ac-5c2f-4704-bb4c-6203384cb7df None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:36.962792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f47ac-5c2f-4704-bb4c-6203384cb7df None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:36.964988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 578/1154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:36.971288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:36.971744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:36.971840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:36.994288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:36.995475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:37.065992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 577/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:37.089536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f47ac-5c2f-4704-bb4c-6203384cb7df None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:37.173978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:37.201032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f47ac-5c2f-4704-bb4c-6203384cb7df None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:37.236263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:37.236817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:37.237100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:37.237366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:37.248313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:37.253272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:37.254378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:37.255644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 579/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:37.260659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:37.353124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:37.361173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc6512-6db3-4bf4-bdee-c98fda33c8b1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:37.396367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:37.396972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:37.397246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:37.397513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:37.398174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:37.398480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:37.399851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:37.407988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 578/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:29:37.414550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:37.526883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab062b-dc8d-4129-8b1c-bada2003fdd6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:37.572670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:37.573302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:37.573577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:37.573840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] PATH_INFO: `/v3/users/676fb1f48bd549778d8ab97b3acd712a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:37.575113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] RBAC: Authorizing `identity:get_user(user_id=676fb1f48bd549778d8ab97b3acd712a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:37.576539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:37.578120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 580/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:37 2022] GET /identity/v3/users/676fb1f48bd549778d8ab97b3acd712a => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:37.584358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:37.690718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baea289-0dbf-4610-af62-af1eb82f8787 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:37.730245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:37.730849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:37.731164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:37.731466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:37.756003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:37.807018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:37.809139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:37.908478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 579/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:37.917864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:37.921070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:38.022140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d497c9c-91f1-4828-b906-412fd4ede9a5 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:38.059239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc40362-6fd4-4823-b731-125598e2d86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:38.059784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40362-6fd4-4823-b731-125598e2d86b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:38.060059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40362-6fd4-4823-b731-125598e2d86b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:38.060330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40362-6fd4-4823-b731-125598e2d86b None admin] PATH_INFO: `/v3/projects/3289e11e660c40aeb3bf4d37313032a8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:38.061569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc40362-6fd4-4823-b731-125598e2d86b None admin] RBAC: Authorizing `identity:get_project(project_id=3289e11e660c40aeb3bf4d37313032a8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:38.063023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc40362-6fd4-4823-b731-125598e2d86b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:38.064649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 581/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:37 2022] GET /identity/v3/projects/3289e11e660c40aeb3bf4d37313032a8 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:38.071818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:38.140409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:38.189508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edee9d-e76c-4ddb-b34e-828941cd7484 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:38.225808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:38.226792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:38.227068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:38.227336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:38.228066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=3289e11e660c40aeb3bf4d37313032a8, user.id=676fb1f48bd549778d8ab97b3acd712a) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:38.228375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:38.229808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fd815-24b9-46bf-ad60-e3b0535976d0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:38.235714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 580/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:38 2022] GET /identity/v3/role_assignments?user.id=676fb1f48bd549778d8ab97b3acd712a&scope.project.id=3289e11e660c40aeb3bf4d37313032a8&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:40.205340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0064-3037-4079-b7ca-51cc94d9ab8a None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:40.206293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0064-3037-4079-b7ca-51cc94d9ab8a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:40.206293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0064-3037-4079-b7ca-51cc94d9ab8a None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:40.208413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:40.214574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:40.215003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:40.215131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:40.240028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:40.241246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:40.313456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:40.346281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0064-3037-4079-b7ca-51cc94d9ab8a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:40.431963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:40.466464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0064-3037-4079-b7ca-51cc94d9ab8a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:40.504814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:40.505410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:40.505687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:40.505955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:40.579519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:40.580959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:40.662430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:40.697177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 583/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 02:29:40.703551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:40.754598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:40.805086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad644dfe-0d0c-48b8-99bd-3a5891560863 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:40.842978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:40.843602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:40.843877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:40.844148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:40.851445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:40.852900 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-367a36b3-6ad7-48ff-a412-97627224c423 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:40.864414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:40.865562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:40.866594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 582/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:29:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:40.871786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:40.975628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bc0da-b91b-4ed6-ab68-9a3dfb2a2980 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:41.009840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:41.010404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:41.010672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:41.010937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:41.011572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:41.011862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:41.013270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff2fc-18d0-4304-b0cb-0add23a3b648 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:41.020152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 584/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 9 02:29:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:42.986129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157e7d-9ac4-429a-b9f8-a3dc561bca8e None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:42.986749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157e7d-9ac4-429a-b9f8-a3dc561bca8e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:42.986749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50157e7d-9ac4-429a-b9f8-a3dc561bca8e None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:42.988063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 583/1167] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:42.994379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f71714-87e5-4cb0-8e65-9209ee94f3bb None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:42.995124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f71714-87e5-4cb0-8e65-9209ee94f3bb None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:42.995259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f71714-87e5-4cb0-8e65-9209ee94f3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:43.020869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e8f71714-87e5-4cb0-8e65-9209ee94f3bb None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:43.022061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e8f71714-87e5-4cb0-8e65-9209ee94f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:43.050433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:43.102288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 585/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:43.125525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157e7d-9ac4-429a-b9f8-a3dc561bca8e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:43.239928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50157e7d-9ac4-429a-b9f8-a3dc561bca8e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:43.249877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:43.275377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2d8dc-4fd4-4413-bf83-a7f5efa10e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:43.275981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2d8dc-4fd4-4413-bf83-a7f5efa10e48 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:43.276256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2d8dc-4fd4-4413-bf83-a7f5efa10e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:43.276526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2d8dc-4fd4-4413-bf83-a7f5efa10e48 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:43.277184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2d8dc-4fd4-4413-bf83-a7f5efa10e48 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:43.278637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2d8dc-4fd4-4413-bf83-a7f5efa10e48 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:43.296127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 584/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Aug 9 02:29:43 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:45.282044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e71ac-4b74-4221-bd57-7a9cb168f6dd None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:45.283041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e71ac-4b74-4221-bd57-7a9cb168f6dd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:45.283041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e71ac-4b74-4221-bd57-7a9cb168f6dd None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:45.285090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 586/1170] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:45.291412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:45.292139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:45.292283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:45.312253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:45.313462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:45.391796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:45.414747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e71ac-4b74-4221-bd57-7a9cb168f6dd None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:45.522696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e71ac-4b74-4221-bd57-7a9cb168f6dd None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:45.557186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff0d2d-f23d-4228-a882-36f60a78d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:45.557728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff0d2d-f23d-4228-a882-36f60a78d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:45.558003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff0d2d-f23d-4228-a882-36f60a78d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:45.558302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff0d2d-f23d-4228-a882-36f60a78d3d5 None admin] PATH_INFO: `/v3/domains/f0177c9b5c0546079ae66abb154aa775` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:45.569150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff0d2d-f23d-4228-a882-36f60a78d3d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f0177c9b5c0546079ae66abb154aa775)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:45.570621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff0d2d-f23d-4228-a882-36f60a78d3d5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:45.572172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 587/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Aug 9 02:29:45 2022] GET /identity/v3/domains/f0177c9b5c0546079ae66abb154aa775 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:45.578189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:45.678986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:45.684704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabce7f-af63-4c24-ade1-38392fa6fbf8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:45.750776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690bec9-e34c-4358-be0b-3e46d1b2de80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:45.751381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690bec9-e34c-4358-be0b-3e46d1b2de80 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:45.751657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690bec9-e34c-4358-be0b-3e46d1b2de80 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:45.751925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690bec9-e34c-4358-be0b-3e46d1b2de80 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:45.752556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690bec9-e34c-4358-be0b-3e46d1b2de80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:45.754001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690bec9-e34c-4358-be0b-3e46d1b2de80 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:45.784515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 586/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:29:45 2022] POST /identity/v3/projects => generated 361 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:47.857187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c468708-030a-4772-b3b3-ee5425452ec5 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:47.858036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c468708-030a-4772-b3b3-ee5425452ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:47.858036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c468708-030a-4772-b3b3-ee5425452ec5 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:47.860263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 588/1174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:47.866578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:47.867319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:47.867479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:47.887986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:47.889167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:47.957160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 587/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:47.982028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c468708-030a-4772-b3b3-ee5425452ec5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:48.015864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:48.115317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c468708-030a-4772-b3b3-ee5425452ec5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:48.150996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa03101-0aa8-478a-a340-586cccbc8076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:48.151641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa03101-0aa8-478a-a340-586cccbc8076 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:48.151919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa03101-0aa8-478a-a340-586cccbc8076 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:48.152174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa03101-0aa8-478a-a340-586cccbc8076 None admin] PATH_INFO: `/v3/domains/f0177c9b5c0546079ae66abb154aa775` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:48.153353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa03101-0aa8-478a-a340-586cccbc8076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f0177c9b5c0546079ae66abb154aa775)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:48.154826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa03101-0aa8-478a-a340-586cccbc8076 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:48.156382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 589/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Aug 9 02:29:47 2022] GET /identity/v3/domains/f0177c9b5c0546079ae66abb154aa775 => generated 267 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:48.162617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:48.249778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:48.273795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6025b-bfb5-4852-ba5b-777359c374a4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:48.308811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb10d2e-8840-432c-bf7e-6a8274d491b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:48.309410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb10d2e-8840-432c-bf7e-6a8274d491b9 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:48.309687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb10d2e-8840-432c-bf7e-6a8274d491b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:48.309958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb10d2e-8840-432c-bf7e-6a8274d491b9 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:48.310633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb10d2e-8840-432c-bf7e-6a8274d491b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:48.312086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb10d2e-8840-432c-bf7e-6a8274d491b9 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:48.335428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 588/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:29:48 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:29:50.358910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907c514-7c6d-4c39-a04b-853e8792d349 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:50.359781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907c514-7c6d-4c39-a04b-853e8792d349 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:50.359781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907c514-7c6d-4c39-a04b-853e8792d349 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:50.361944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 590/1178] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:50.368226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:50.368895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:50.369049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:50.389470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:50.390703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:50.464543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:50.494541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907c514-7c6d-4c39-a04b-853e8792d349 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:50.596741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:50.608767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907c514-7c6d-4c39-a04b-853e8792d349 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:50.643663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:50.644217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:50.644499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:50.644773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:50.655104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:50.659811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:50.660898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:50.662745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 591/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:50.671036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:50.748912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:50.789892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305b396-270c-4723-978b-2da3ff412c12 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:50.825006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:50.825597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:50.825869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:50.826136 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:50.826830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:50.827140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:50.828508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:50.836502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 590/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:50.842651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:50.946964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83b8cf-19e6-44b5-bb85-c9310ee088e1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:50.981626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:50.982257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:50.982540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:50.982788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] PATH_INFO: `/v3/users/b5cc22d79c3e49d79b68eb6ab4d346c7` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:50.995816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] RBAC: Authorizing `identity:get_user(user_id=b5cc22d79c3e49d79b68eb6ab4d346c7)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:50.997322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:50.998929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 592/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:50 2022] GET /identity/v3/users/b5cc22d79c3e49d79b68eb6ab4d346c7 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:51.004737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:51.111722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef35400-f96e-44af-bd0f-01561e14fca0 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:51.141460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:51.150414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:51.151324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:51.151754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:51.152124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:51.228960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:51.230381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:51.329531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 591/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:51.335936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:51.348663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:51.446283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b297-c222-405d-bbb5-409fef16016b None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:51.482793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c5765-6af7-4f0a-816f-2f047303b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:51.483363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c5765-6af7-4f0a-816f-2f047303b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:51.483647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c5765-6af7-4f0a-816f-2f047303b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:51.483923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c5765-6af7-4f0a-816f-2f047303b4d2 None admin] PATH_INFO: `/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:51.485109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c5765-6af7-4f0a-816f-2f047303b4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8d409ccc65bd4edfb79bbff5e4f9bc57)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:51.486575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c5765-6af7-4f0a-816f-2f047303b4d2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:51.488202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 593/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:51 2022] GET /identity/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:51.494586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:51.523792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:51.594353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61194211-f1c3-406f-8c3d-753bc98d088d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:51.629490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:51.630086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:51.630400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:51.630674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:51.631409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=8d409ccc65bd4edfb79bbff5e4f9bc57, user.id=b5cc22d79c3e49d79b68eb6ab4d346c7) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:51.631716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:51.633166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adadd9d-54ec-4722-ae74-9842f04494c1 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:51.638695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 592/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:51 2022] GET /identity/v3/role_assignments?user.id=b5cc22d79c3e49d79b68eb6ab4d346c7&scope.project.id=8d409ccc65bd4edfb79bbff5e4f9bc57&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:53.572874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa64b37-346e-4432-8a54-d79b23d3b9e2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:53.573431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa64b37-346e-4432-8a54-d79b23d3b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:53.573431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa64b37-346e-4432-8a54-d79b23d3b9e2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:53.574778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 594/1186] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:53.581194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:53.581884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:53.582072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:53.611121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:53.612331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:53.682974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:53.711686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa64b37-346e-4432-8a54-d79b23d3b9e2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:53.797324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:53.826866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa64b37-346e-4432-8a54-d79b23d3b9e2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:53.861212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:53.861834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:53.862114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:53.862420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:53.872939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:53.877652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:53.878752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:29:53.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 595/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:53.885105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:53.912404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:54.002309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d42c-ec5d-4f3d-b391-f1d5b522ea90 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:54.037795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:54.038423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:54.038705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:54.038976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:54.039640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:54.039950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:54.041330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:54.049431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 594/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:54.055619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:54.157609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e843-05c3-4fc2-ab34-f0d24bab2eef None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:54.193782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:54.194425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:54.194699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:54.194967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] PATH_INFO: `/v3/users/b5cc22d79c3e49d79b68eb6ab4d346c7` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:54.196184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] RBAC: Authorizing `identity:get_user(user_id=b5cc22d79c3e49d79b68eb6ab4d346c7)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:54.197608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:54.199191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 596/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:54 2022] GET /identity/v3/users/b5cc22d79c3e49d79b68eb6ab4d346c7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:54.205877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:54.298625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:54.321912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe1df-47f7-4a82-b287-5542edf915ba None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:54.360451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f7980-1059-4e57-9e63-c1d17f7b1e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:54.361076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f7980-1059-4e57-9e63-c1d17f7b1e88 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:54.361352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f7980-1059-4e57-9e63-c1d17f7b1e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:54.361618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f7980-1059-4e57-9e63-c1d17f7b1e88 None admin] PATH_INFO: `/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:54.362835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f7980-1059-4e57-9e63-c1d17f7b1e88 None admin] RBAC: Authorizing `identity:get_project(project_id=8d409ccc65bd4edfb79bbff5e4f9bc57)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:54.364254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f7980-1059-4e57-9e63-c1d17f7b1e88 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:54.365915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 595/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:54 2022] GET /identity/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:54.375907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:54.389175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:54.497135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad675544-2729-4c93-bd4a-68d2f7fda9ad None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:54.531436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aedef7-3ee4-4caf-9534-5a3491f5b5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:54.532093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aedef7-3ee4-4caf-9534-5a3491f5b5dc None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:54.532371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aedef7-3ee4-4caf-9534-5a3491f5b5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:54.532646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aedef7-3ee4-4caf-9534-5a3491f5b5dc None admin] PATH_INFO: `/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57/users/b5cc22d79c3e49d79b68eb6ab4d346c7/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:54.534720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aedef7-3ee4-4caf-9534-5a3491f5b5dc None admin] RBAC: Authorizing `identity:create_grant(project_id=8d409ccc65bd4edfb79bbff5e4f9bc57, user_id=b5cc22d79c3e49d79b68eb6ab4d346c7, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:54.536159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aedef7-3ee4-4caf-9534-5a3491f5b5dc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:54.552911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 597/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:29:54 2022] PUT /identity/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57/users/b5cc22d79c3e49d79b68eb6ab4d346c7/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:29:56.508683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937402fc-69a9-4134-80d5-a1f1029c973f None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:56.509474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937402fc-69a9-4134-80d5-a1f1029c973f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:56.509620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937402fc-69a9-4134-80d5-a1f1029c973f None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:56.511818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 596/1193] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:29:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:29:56.517493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:56.517914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:56.518028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:56.537691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:29:56.538908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:29:56.599974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:56.609318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 598/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:29:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:29:56.632526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937402fc-69a9-4134-80d5-a1f1029c973f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:56.772454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937402fc-69a9-4134-80d5-a1f1029c973f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:56.807542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:56.808168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:56.808449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:56.808722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:56.819916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:29:56.824598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:29:56.825655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:29:56.826595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 597/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:29:56.832414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:56.942707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427083d-3c41-4bd9-abac-d3a1f22e7f59 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:56.982186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:56.982794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:56.983075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:56.983349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:56.984026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:56.984330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:56.985703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:56.993969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 599/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:57.000289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:57.016961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:57.103209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506a82a-5b64-4a72-a67b-9c68afe75a21 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:57.138659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:57.139305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:57.139601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:57.139879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] PATH_INFO: `/v3/users/b5cc22d79c3e49d79b68eb6ab4d346c7` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:57.141079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] RBAC: Authorizing `identity:get_user(user_id=b5cc22d79c3e49d79b68eb6ab4d346c7)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:57.142675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:57.144370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 598/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 9 02:29:56 2022] GET /identity/v3/users/b5cc22d79c3e49d79b68eb6ab4d346c7 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:57.150495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:57.209658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:57.262561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab93822-52b9-4765-97be-0c3ec4e6859b None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:57.297464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:57.298063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:57.298380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:57.298652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:57.373112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:57.374977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:57.443178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:57.489669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 600/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:29:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:29:57.496571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:57.601334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:29:57.621546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da20d98-5828-4837-8fc5-eca4cd6365bd None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:57.655555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c402-9069-48c3-b206-788799f60368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:57.656113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c402-9069-48c3-b206-788799f60368 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:57.656386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c402-9069-48c3-b206-788799f60368 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:57.656650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c402-9069-48c3-b206-788799f60368 None admin] PATH_INFO: `/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:57.657878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201c402-9069-48c3-b206-788799f60368 None admin] RBAC: Authorizing `identity:get_project(project_id=8d409ccc65bd4edfb79bbff5e4f9bc57)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:57.659554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201c402-9069-48c3-b206-788799f60368 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:57.661177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 599/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 9 02:29:57 2022] GET /identity/v3/projects/8d409ccc65bd4edfb79bbff5e4f9bc57 => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:29:57.667746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:29:57.769770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336b754-e177-4b3c-9039-5b5a352639ba None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:29:57.804385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:29:57.804972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:29:57.805246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:29:57.805524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:29:57.806270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=8d409ccc65bd4edfb79bbff5e4f9bc57, user.id=b5cc22d79c3e49d79b68eb6ab4d346c7) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:29:57.806580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:29:57.808000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043b3d-33ee-447a-a7cd-3ce87f540311 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:29:57.813859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 601/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:29:57 2022] GET /identity/v3/role_assignments?user.id=b5cc22d79c3e49d79b68eb6ab4d346c7&scope.project.id=8d409ccc65bd4edfb79bbff5e4f9bc57&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:30:07.650565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=25 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:07.811436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:52.858211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af720db-b640-4446-aaac-a9afb85a9f54 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:52.858823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af720db-b640-4446-aaac-a9afb85a9f54 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:52.858823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af720db-b640-4446-aaac-a9afb85a9f54 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:52.860480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 600/1201] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:30:52.867052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:52.867548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:52.867775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:52.890552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:30:52.891760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:30:52.990261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 602/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:30:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:30:53.017147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af720db-b640-4446-aaac-a9afb85a9f54 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:53.159417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af720db-b640-4446-aaac-a9afb85a9f54 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:53.204404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:53.205067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:53.205345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:53.205626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:53.219607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:53.221237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:30:53.238059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:30:53.244374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:30:53.245572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:30:53.245572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:30:53.245572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:30:53.245572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 601/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:30:53.245882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:53.368997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f76e81-b561-4806-ab7a-ddf2dd28ebab None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:53.410493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:53.421099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:53.421359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:53.421733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:53.422018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:53.422765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:53.423072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:53.424462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cb173-9510-4e3f-947c-600114b1809a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:53.442521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 603/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:30:53.444177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:53.514521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:53.575256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1e805-291a-42b7-90d1-529941aa45ca None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:53.625708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4fea-fb7f-44b7-994c-aabdf634a7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:53.626618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4fea-fb7f-44b7-994c-aabdf634a7d5 None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:53.627116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4fea-fb7f-44b7-994c-aabdf634a7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:53.627538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4fea-fb7f-44b7-994c-aabdf634a7d5 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:53.628406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4fea-fb7f-44b7-994c-aabdf634a7d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:53.629927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4fea-fb7f-44b7-994c-aabdf634a7d5 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:53.653921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 602/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 9 02:30:53 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:30:55.953351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3ab30-06c7-4c18-a962-cb561ebe9b8e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:55.953989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3ab30-06c7-4c18-a962-cb561ebe9b8e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:55.954072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3ab30-06c7-4c18-a962-cb561ebe9b8e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:55.955778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 604/1206] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:30:55.961906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:55.962621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:55.962621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:55.985223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:30:55.986488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:30:56.063472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 603/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:30:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:30:56.088194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3ab30-06c7-4c18-a962-cb561ebe9b8e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:56.210407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3ab30-06c7-4c18-a962-cb561ebe9b8e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:56.250484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:56.251131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:56.251413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:56.251737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:56.263555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:30:56.272519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:30:56.273693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:30:56.274932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 605/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:30:56.275669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:56.321293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:56.391702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca09d-deff-471d-9b1f-17bd43e20574 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:56.432538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:56.433217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:56.433492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:56.433832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:56.434568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:56.434877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:56.436310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:56.449395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 604/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:30:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:30:56.451093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:56.507032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:56.561726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3689e-bf9e-4b86-965c-d06388dde1ed None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:56.601347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:56.602092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:56.602435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:56.602768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:56.614314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:56.615961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:30:56.632326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:30:56.633486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:30:56.634529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:30:56.634529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:30:56.634529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:30:56.634529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 606/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:30:56.639826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:56.752901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:56.755764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73621-a923-496a-b1cd-4ed8fd0a5c3e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:56.795016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:56.795611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:56.795893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:56.796167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:56.796884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:56.797225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:56.798666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:56.812238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 605/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:30:56.818233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:56.938732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917174-f6f1-4e7e-8161-33777a360fdc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:56.940163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:56.980719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:56.981588 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:56.982012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:56.982301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:56.991530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:56.993173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:30:57.006365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:30:57.007464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:30:57.008579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 607/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:30:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:30:57.013942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:57.124210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab25c-14ad-41fc-b2f9-22426b8f30e2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:57.161754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:57.162472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:57.162774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:57.163031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:57.163811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:57.164123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:57.165619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:57.187457 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 606/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 9 02:30:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:30:57.194802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:57.321123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c975f-3ffb-422d-9212-c33c1c380c81 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:57.363589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:57.364358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:57.364827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:57.365188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:57.403338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:57.453016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:57.454567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:57.568506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 608/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:30:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:30:57.574668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:57.594966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:57.686246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98337de-6bf4-4856-afb9-a901a611eec8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:57.723328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:57.724152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:57.724574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:57.724914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:57.738928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:57.740579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:30:57.756903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:30:57.758010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:30:57.759052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:30:57.759052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:30:57.759052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:30:57.763486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 607/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:30:57.764738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:57.834448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:57.879714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49051714-c5ae-422c-a1fd-c125e4519bf9 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:57.920217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:57.921028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:57.921413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:57.921734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:57.922638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:57.922990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:57.924488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:57.938549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 609/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:30:57.944128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:58.019178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:58.052673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f0458-1077-4ada-af2f-981c8382588c None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:58.088209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:58.088851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:58.089169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:58.089413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:58.096764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:58.097880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:30:58.109632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:30:58.110860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:30:58.111835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 608/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:30:58.116664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:58.222955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d0b17-3f98-48dc-827b-e745b9a1d5b3 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:58.246175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:58.257598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:58.258236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:58.258578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:58.258857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:58.259554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:58.259857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:58.261293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d50c1-c36e-486e-8162-b2c6700a07f3 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:58.273994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 610/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:30:58.283387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:30:58.419395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708121-8aee-4e77-9f9f-69961f07a474 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:30:58.452490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:30:58.461262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:30:58.462219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:30:58.462620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:30:58.462904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:30:58.464071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=dea38c5b41e5432d9f94f3248824e262) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:30:58.464539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:30:58.466440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8dd2f-4421-47c1-9824-fe79424f83ee None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:30:58.486291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 609/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:30:58 2022] GET /identity/v3/role_assignments?user.id=dea38c5b41e5432d9f94f3248824e262&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:00.660909 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59737615-a5d6-45ec-920d-4cc8e535cda8 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:00.661572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59737615-a5d6-45ec-920d-4cc8e535cda8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:00.661572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59737615-a5d6-45ec-920d-4cc8e535cda8 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:00.662887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 611/1220] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:00.668194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:00.668727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:00.668846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:00.689109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:00.690446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:00.767630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 610/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:00.794302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59737615-a5d6-45ec-920d-4cc8e535cda8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:00.901153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59737615-a5d6-45ec-920d-4cc8e535cda8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:00.937851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:00.938438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:00.938719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:00.938991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:00.949269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:31:00.954006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:00.955397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:00.956506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 612/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:31:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:00.961416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:01.066767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19f1be-54b1-4dd4-aabf-fec33ff1f2d5 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:01.093218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:01.105192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:01.105861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:01.106142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:01.106442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:01.107117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] RBAC: Adding query filter params (name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:01.107485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:01.108970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:01.121993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 611/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:31:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:31:01.123295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:01.174768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:01.235950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbae3f-46ae-4d46-a21a-6b1ea13f2ed4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:01.274219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:01.274992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:01.275381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:01.275655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:01.286598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:01.288044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:01.301889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:01.303091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:01.304254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:01.304254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:01.304254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:01.304254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 613/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:01.311081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:01.418367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885aa5b-b3bc-4320-a284-ad442c825789 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:01.456077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:01.456710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:01.457038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:01.457353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:01.458082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:01.458447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:01.459927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:01.473242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 612/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:01.479674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:01.580415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:01.587172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f49ac-0725-4465-bb0f-989568eb1518 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:01.625644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:01.626306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:01.626628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:01.626984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:01.634879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:01.636439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:31:01.648673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:01.649903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:01.651086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:01.654319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 614/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:01.655819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:01.708138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:01.766258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef59a-de81-4e29-b323-d187136d9789 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:01.800361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:01.800978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:01.801255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:01.801522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:01.802290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:01.802594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:01.804020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:01.821528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 613/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 9 02:31:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:01.827769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:01.942062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b954d-4b1a-4312-99ae-7a30658564e7 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:01.976996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:01.977674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:01.978068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:01.978397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:01.989852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:01.991345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:02.005663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:02.007017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:02.008683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:02.008683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:02.008683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:02.008683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 615/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:02.013639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:02.124099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3bd30-8b14-4d88-99e4-3ca78ebca83b None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:02.126503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:02.161868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:02.162526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:02.162890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:02.163168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:02.163827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:02.164130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:02.165519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:02.177731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 614/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:31:02.184557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:02.195408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:02.297290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb595-2717-4dc0-9de0-4b966f6d96b2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:02.332546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:02.333186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:02.333465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:02.333738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:02.342095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:02.343576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:31:02.355737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:02.356889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:02.358256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 616/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:31:02.365545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:02.481663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece7a46-1208-4074-bafc-1546aaf17d25 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:02.517686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:02.518312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:02.518592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:02.518860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:02.519551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:02.519855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:02.521451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992f9ab-1051-4004-b2ae-a2e91cdc61c6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:02.538178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 615/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:02.540555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:02.612479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:02.650276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906616fa-9d08-43c5-b760-923c1d31953e None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:02.685504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b9b61-7b1b-4a5e-b757-581842a5102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:02.686268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b9b61-7b1b-4a5e-b757-581842a5102f None admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:02.686577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b9b61-7b1b-4a5e-b757-581842a5102f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:02.686895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b9b61-7b1b-4a5e-b757-581842a5102f None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/dea38c5b41e5432d9f94f3248824e262/roles/1d7b520e46b9400eacfddde5dce1db4a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:02.704326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b9b61-7b1b-4a5e-b757-581842a5102f None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=dea38c5b41e5432d9f94f3248824e262, role_id=1d7b520e46b9400eacfddde5dce1db4a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:02.705856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b9b61-7b1b-4a5e-b757-581842a5102f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:02.719850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 617/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:31:02 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/dea38c5b41e5432d9f94f3248824e262/roles/1d7b520e46b9400eacfddde5dce1db4a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:31:04.709884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165fdd2-6a11-4ffd-83ea-b9705b91a726 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:04.710466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165fdd2-6a11-4ffd-83ea-b9705b91a726 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:04.710466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165fdd2-6a11-4ffd-83ea-b9705b91a726 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:04.711756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 616/1233] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:04.717008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06964434-a883-42a0-bddd-56890af93993 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:04.717415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06964434-a883-42a0-bddd-56890af93993 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:04.717558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06964434-a883-42a0-bddd-56890af93993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:04.736930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-06964434-a883-42a0-bddd-56890af93993 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:04.738131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-06964434-a883-42a0-bddd-56890af93993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:04.814060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 618/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:04.841214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165fdd2-6a11-4ffd-83ea-b9705b91a726 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:04.869950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:04.947909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165fdd2-6a11-4ffd-83ea-b9705b91a726 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:04.985115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:04.985668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:04.985950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:04.986254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:04.998232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:31:05.003010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:05.004110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:05.005263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 617/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 9 02:31:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:05.011143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06964434-a883-42a0-bddd-56890af93993 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:05.057751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:05.112605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06964434-a883-42a0-bddd-56890af93993 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:05.149654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f975d1-0457-4436-b24c-049947712924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:05.150987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f975d1-0457-4436-b24c-049947712924 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:05.151313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f975d1-0457-4436-b24c-049947712924 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:05.151540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f975d1-0457-4436-b24c-049947712924 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:05.152448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f975d1-0457-4436-b24c-049947712924 None admin] RBAC: Adding query filter params (name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:05.152889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f975d1-0457-4436-b24c-049947712924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:05.154928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f975d1-0457-4436-b24c-049947712924 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:05.165134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 619/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 9 02:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:05.171282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:05.269899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:05.286679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef17f1-8d5c-49bf-9ae3-924ed5ad4331 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:05.321046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:05.321679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:05.321952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:05.322243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:05.332149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:05.333627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:05.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:05.348754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:05.355062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:05.355062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:05.355062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:05.355062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 618/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:05.355374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f975d1-0457-4436-b24c-049947712924 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:05.466420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f975d1-0457-4436-b24c-049947712924 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:05.475201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:05.504165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:05.504870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:05.505157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:05.505446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:05.506075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:05.506409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:05.507781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:05.519528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 620/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:05.525646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:05.632225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a662c-d622-4d6f-8806-dc438ccb4fd2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:05.667514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:05.668177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:05.668500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:05.668731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:05.671725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:05.678261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:05.680441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:31:05.691488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:05.692628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:05.693611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 619/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:05.699265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:05.806476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b702e-3ab6-4297-97d2-d6f7cf16cca6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:05.841165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:05.841869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:05.842146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:05.842446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:05.843187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:05.843488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:05.844925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:05.862950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 621/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 9 02:31:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:05.869162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:05.983746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5275a-5c8a-4e1d-a60b-8d4b8c517b11 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:06.023251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:06.023867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:06.024151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:06.024424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:06.089912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:06.097112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:06.098683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:06.210360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 620/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:31:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:31:06.216751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:06.250031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:06.338425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692405f-29ee-41c6-b256-e7e483c880ea None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:06.378445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:06.379162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:06.379440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:06.379707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:06.390398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:06.392051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:06.409065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:06.410272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:06.411365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:06.411365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:06.411365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:06.411365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 622/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:06.416709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:06.526133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:06.532959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78d77-81d1-450d-84e5-d74f79583939 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:06.569018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:06.569692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:06.569983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:06.570344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:06.571009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:06.571297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:06.572659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:06.589409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 621/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:06.590778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:06.677072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:06.699991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce85d1a-4a11-4d41-befd-ee2403a952a8 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:06.741750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:06.742421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:06.742687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:06.742954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:06.751055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:06.752497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:31:06.771310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:06.772517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:06.773461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 623/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:06.778581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:06.890586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad328-2c85-4427-af26-3074fa259625 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:06.933634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:06.934241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:06.934581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:06.934856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:06.935788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:06.936151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:06.937953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beb1a7-1871-4324-a8e5-cb80dfce1a35 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:06.953280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 622/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:06.960439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:07.063187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8c731-4c2e-448e-b4a5-bbdc31068139 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:07.095878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:07.096500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:07.096777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:07.097039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:07.097768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] RBAC: Adding query filter params (role.id=1d7b520e46b9400eacfddde5dce1db4a, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=dea38c5b41e5432d9f94f3248824e262) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:07.098075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:07.099533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124dc51-4689-4754-a129-a393748402c1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:07.105876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 624/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:31:06 2022] GET /identity/v3/role_assignments?user.id=dea38c5b41e5432d9f94f3248824e262&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=1d7b520e46b9400eacfddde5dce1db4a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:09.405461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6d3d1-0a72-4b8b-a6a8-8fd26a3fd183 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:09.406041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6d3d1-0a72-4b8b-a6a8-8fd26a3fd183 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:09.406145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6d3d1-0a72-4b8b-a6a8-8fd26a3fd183 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:09.407812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 623/1247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:09.414565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:09.415039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:09.415172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:09.421796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:09.436232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:09.437689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:09.517882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:09.544517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6d3d1-0a72-4b8b-a6a8-8fd26a3fd183 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:09.565181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:09.656959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6d3d1-0a72-4b8b-a6a8-8fd26a3fd183 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:09.693442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:09.694109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:09.694407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:09.694680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:09.705297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:31:09.710045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:09.711292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:09.712289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 624/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:31:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:09.717666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:09.828485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682d48b-f7da-46ee-9623-97b0b11ea5ed None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:09.863611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:09.869889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:09.870678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:09.871080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:09.871411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:09.872340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:09.872757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:09.874675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:09.884311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 626/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:31:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:09.890859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:09.977454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:10.002251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e32a3-f175-4ff4-b695-420161c0f2a2 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:10.038707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:10.039351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:10.039633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:10.039902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:10.052042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:10.053573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:10.068802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:10.070036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:10.076046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:10.076046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:10.076046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:10.076046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 625/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:31:10.076440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:10.193745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5301c8-33db-4450-ae5f-7f10755289f1 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:10.230027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:10.230590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:10.230885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:10.231156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:10.231811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:10.232112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:10.233550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:10.248553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 627/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:10.254755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:10.362646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551112c2-6d59-4040-9baa-7c3c3b693ba8 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:10.392716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:10.401717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:10.402396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:10.402676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:10.404304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:10.411452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:10.412973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:31:10.425016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:10.426250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:10.427468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 626/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:10.432165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:10.502581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:10.555035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888717c-647a-4b3e-b0e5-7a44645c1ab5 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:10.593836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:10.594509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:10.594817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:10.595116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:10.595927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:10.596286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:10.597740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:10.616718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 628/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 9 02:31:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:10.622942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:10.733370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12b33a-af0e-4535-b3b4-9ca0953a5a42 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:10.770365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:10.770904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:10.771187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:10.771448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:10.837398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:10.859917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:10.863178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:10.975688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 627/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:31:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:31:10.982895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:11.092516 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013b043-c4d1-4df9-8f53-a1d6ed5a3a53 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:11.107165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:11.130284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:11.130924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:11.131211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:11.131564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:11.144096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:11.145569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:11.165582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:11.166888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:11.167838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:11.167838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:11.167838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:11.167838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 629/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:11.173770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:11.270572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:11.285016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef41033-11de-42bf-8439-c362d578de27 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:11.321188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:11.321799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:11.322077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:11.322396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:11.323042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:11.323367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:11.324763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:11.336083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 628/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:11.342596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:11.465902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd67b42-e726-4623-88e4-ff1fcc1b3da6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:11.501995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:11.502776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:11.503150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:11.503361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:11.511200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:11.512771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:31:11.525692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:11.527001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:11.528337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 630/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:31:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:11.533872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:11.646211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e16104-1106-4f41-b252-e1b51aa8a2f7 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:11.688890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:11.689472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:11.689748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:11.690013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:11.690743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:11.691130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:11.692703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51407b-8432-493d-858a-1956ea108d39 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:11.706888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 629/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:11.713223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:11.742019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:11.818741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfbcd7-c101-4db2-9250-7e9097dc3351 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:11.857708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:11.858549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:11.859040 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:11.859356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:11.860199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=dea38c5b41e5432d9f94f3248824e262) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:11.860507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:11.862067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d5f3c-ae05-4fa2-bb28-62cbcaa9329e None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:11.869501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 631/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:31:11 2022] GET /identity/v3/role_assignments?user.id=dea38c5b41e5432d9f94f3248824e262&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:31:13.953201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f2699-06a8-409d-a593-705de77895eb None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:13.953765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f2699-06a8-409d-a593-705de77895eb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:13.953765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f2699-06a8-409d-a593-705de77895eb None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:13.955171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 630/1261] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:13.961439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:13.962128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:13.962308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:13.989738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:13.991009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:14.059192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 632/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:14.094087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f2699-06a8-409d-a593-705de77895eb None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:14.106767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:14.214094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f2699-06a8-409d-a593-705de77895eb None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:14.246634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:14.247260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:14.247537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:14.247808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:14.258708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:31:14.263411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:14.264515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:14.265734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 631/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:31:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:14.271522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:14.292931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:14.374397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fe016-1314-4aed-bd66-f21c8d6faece None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:14.412951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:14.413176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:14.413458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:14.413725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:14.414419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:14.414731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:14.416089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:14.425358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 633/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:31:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:31:14.431531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:14.502763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:14.541085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd27f32-6ce2-493a-80ae-a956f303eb44 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:14.576224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:14.576873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:14.577150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:14.577422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:14.587742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:14.589185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:14.603811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:14.605264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:14.606742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:14.606742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:14.606742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:14.606742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 632/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:14.612032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:14.697418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:14.719585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc0f62-6c41-4e60-898c-4a888c7dc1f6 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:14.754981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:14.755500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:14.755772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:14.756043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:14.756688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:14.756992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:14.758384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:14.769965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 634/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:14.778512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:14.883471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554f4c9-4b54-4f02-b417-0efaf1f4cabd None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:14.902940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:14.918337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:14.918958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:14.919234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:14.919508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:14.927901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:14.929354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:31:14.940568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:14.942121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:14.943951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 633/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:14.949594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:15.075196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cb797-63de-4c4b-b4b2-a3ec9afb356a None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:15.110908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:15.111507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:15.111784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:15.112056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:15.112779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:15.113089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:15.114544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:15.123223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:15.131787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 635/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 9 02:31:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:15.138532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:15.248149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243943e-73c9-4a2e-a9a5-f3df3a8f7ab6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:15.280110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:15.280758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:15.281031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:15.281298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:15.291884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:15.293310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:15.307439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:15.308557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:15.309620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:15.309620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:15.309620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:15.309620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 634/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:15.317273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:15.433023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d3c77-ad9f-4d0d-b88e-8dc070816f15 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:15.468437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:15.469034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:15.469309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:15.469577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:15.470210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:15.470534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:15.471859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:15.483378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 636/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:15.492048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:15.524676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:15.602589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e570a7-fe21-49a2-8844-e8d82d38f19e None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:15.639818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:15.640434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:15.640705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:15.640974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:15.647823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:15.649255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:31:15.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:15.661507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:15.663078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 635/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:15.670482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:15.733826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:15.776467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72395-c540-4bcd-be4c-bfbcb22dab61 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:15.809734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:15.810365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:15.810679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:15.810951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:15.811639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:15.811947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:15.813375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa7f76-9e9c-4ec3-a6b4-01cdcee107a7 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:15.825956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 637/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:15.835100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:15.933045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:15.943702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ad98e-2f9e-4c3c-b2cd-4d92b979cc4d None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:15.979802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737680ce-5ed6-44d5-9aba-4d9c1a9dd80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:15.980461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737680ce-5ed6-44d5-9aba-4d9c1a9dd80b None admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:15.980740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737680ce-5ed6-44d5-9aba-4d9c1a9dd80b None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:15.981016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737680ce-5ed6-44d5-9aba-4d9c1a9dd80b None admin] PATH_INFO: `/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/dea38c5b41e5432d9f94f3248824e262/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:15.983347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737680ce-5ed6-44d5-9aba-4d9c1a9dd80b None admin] RBAC: Authorizing `identity:create_grant(project_id=9deddd84460e4ee9af2f645e34eda375, user_id=dea38c5b41e5432d9f94f3248824e262, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:15.984789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737680ce-5ed6-44d5-9aba-4d9c1a9dd80b None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:15.999775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 636/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 9 02:31:15 2022] PUT /identity/v3/projects/9deddd84460e4ee9af2f645e34eda375/users/dea38c5b41e5432d9f94f3248824e262/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:31:17.979886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048a41-388a-4331-b198-59297908e6a1 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:17.980474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048a41-388a-4331-b198-59297908e6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:17.980474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048a41-388a-4331-b198-59297908e6a1 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:17.981774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 638/1274] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:17.988325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:17.989039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:17.989192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:18.012899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:18.014116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:18.084485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:18.111087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048a41-388a-4331-b198-59297908e6a1 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:18.232323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048a41-388a-4331-b198-59297908e6a1 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:18.254373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:18.274920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:18.275556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:18.275834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:18.276107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:18.287843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 02:31:18.292558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:18.293645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:18.295432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 639/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 9 02:31:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:18.300473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:18.382449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:18.403066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadcfac-28c9-474c-9e1e-d388f9f261e3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:18.440578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:18.441191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:18.441474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:18.441748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:18.442450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:18.442734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:18.444132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:18.452354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 638/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 9 02:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:18.461144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:18.566268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59cf3f-9e88-4e81-840b-ff18780673dc None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:18.601388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:18.602011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:18.602317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:18.602593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:18.613296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:18.614782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:18.629546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:18.630691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:18.631738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:18.631738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:18.631738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:18.631738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 640/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:18.637291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:18.750823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8808-a3d6-4c23-a72a-d48ddca155b6 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:18.785380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:18.785986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:18.786297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:18.786570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:18.787209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:18.787524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:18.788882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:18.805454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 639/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:18.811507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:18.843320 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:18.924837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed36d82-5adf-40d8-b1a8-8e781017e2b2 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:18.959183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:18.959804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:18.960088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:18.960358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:18.968120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:18.969554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 02:31:18.981236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:18.982922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:18.984769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 641/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:18.988952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:18.992914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:19.090993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d38d0-bd8c-4b8d-81eb-8b2add2064bf None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:19.125836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:19.126397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:19.126680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:19.126950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:19.127677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:19.127980 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:19.129411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:19.146181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 640/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 9 02:31:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:19.154007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:19.257141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:19.268749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51ddde-1234-405c-a1d7-361549c41c4b None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:19.306883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:19.307554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:19.307833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:19.308100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:19.376803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:19.378147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:19.470681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:19.481722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 9 02:31:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 02:31:19.490530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:19.558551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:19.601710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0807c-1f71-4dc9-a6aa-2f8a18954222 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:19.646871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:19.647438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:19.647719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:19.647992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:19.658514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:19.659964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 02:31:19.674630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:19.675745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 02:31:19.677334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 02:31:19.677334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 02:31:19.677334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:19.677334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 641/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 02:31:19.684336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:19.784949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51259b-fcde-46e2-a3b1-a242c4417f7f None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:19.826706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:19.827312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:19.827589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:19.827862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:19.828567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:19.828874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:19.830224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:19.841723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 643/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 9 02:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:31:19.848279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:19.960114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cd3d1-9444-40e2-a834-596d2f099630 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:19.973720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:19.995861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:19.996450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:19.996753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:19.997023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:20.004023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:20.005490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 02:31:20.017677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:20.018737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: ERROR keystone.server.flask.application Aug 09 02:31:20.019937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 642/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 9 02:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:20.026644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:20.079364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:20.129234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b99bef-d404-4313-be54-fb9453923401 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:20.163199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:20.163982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:20.164269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:20.164385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:20.165004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:20.165370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:20.166778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50696884-290d-475d-8bdd-1daa9c1ede64 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:20.179557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 644/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 9 02:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:20.188299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:20.298798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf9ab-48c3-4314-af2e-cd1d44a1c985 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:20.333324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:20.333921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:20.334197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:20.334494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:20.335225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] RBAC: Adding query filter params (role.id=4483761ca5a0414abb0dff9144ea2ce5, scope.project.id=9deddd84460e4ee9af2f645e34eda375, user.id=dea38c5b41e5432d9f94f3248824e262) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:20.335530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:20.336969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fd97e-3a3f-4a64-932e-90ca0785f2bb None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:20.343363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 643/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 9 02:31:20 2022] GET /identity/v3/role_assignments?user.id=dea38c5b41e5432d9f94f3248824e262&scope.project.id=9deddd84460e4ee9af2f645e34eda375&role.id=4483761ca5a0414abb0dff9144ea2ce5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:22.341365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f58cc4-64f2-4fa9-ac82-9942ced38cd2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:22.341967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f58cc4-64f2-4fa9-ac82-9942ced38cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:22.341967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f58cc4-64f2-4fa9-ac82-9942ced38cd2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:22.343300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 645/1288] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:22.349604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:22.350346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:22.350492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:22.378392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:22.379595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:22.413364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:22.452609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 644/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:22.474836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f58cc4-64f2-4fa9-ac82-9942ced38cd2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:22.606366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f58cc4-64f2-4fa9-ac82-9942ced38cd2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:22.616972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:22.650607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:22.651236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:22.651518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:22.651789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:22.652368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:22.653730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:22.662699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:22.663920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:22.665171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 646/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:31:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:22.669917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:22.780323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46029a9c-14f6-4b22-832c-5b4b692985c9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:22.815322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:22.815925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:22.816201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:22.816471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:22.817098 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:22.817402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:22.818813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0e98c-42dc-464b-91c6-fbe9b11f6542 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:22.823714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 645/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 9 02:31:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:22.831609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:22.947889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702049e-8cf3-48bb-9550-21e2db7c5b08 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:22.983555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee538804-705f-40d9-913d-211634145a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:22.984146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee538804-705f-40d9-913d-211634145a24 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:22.984497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee538804-705f-40d9-913d-211634145a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:22.984717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee538804-705f-40d9-913d-211634145a24 None admin] PATH_INFO: `/v3/services` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:22.985339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee538804-705f-40d9-913d-211634145a24 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:22.985631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee538804-705f-40d9-913d-211634145a24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:22.987020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee538804-705f-40d9-913d-211634145a24 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:22.991990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 647/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 9 02:31:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:24.960820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de600d47-0320-4db9-9bd1-256635655599 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:24.961407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de600d47-0320-4db9-9bd1-256635655599 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:24.961407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de600d47-0320-4db9-9bd1-256635655599 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:24.962778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 646/1293] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:24.969131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97251-a98f-433b-a301-0dc15f8e5f9b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:24.969818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97251-a98f-433b-a301-0dc15f8e5f9b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:24.969999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97251-a98f-433b-a301-0dc15f8e5f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:24.994916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f5c97251-a98f-433b-a301-0dc15f8e5f9b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:24.996130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f5c97251-a98f-433b-a301-0dc15f8e5f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:25.027269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:25.067076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 648/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:25.089712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de600d47-0320-4db9-9bd1-256635655599 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:25.124605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:25.201985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de600d47-0320-4db9-9bd1-256635655599 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:25.242387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78842f2-0381-4660-8b38-4f55feabecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:25.242926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78842f2-0381-4660-8b38-4f55feabecfc None admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:25.243202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78842f2-0381-4660-8b38-4f55feabecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:25.243474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78842f2-0381-4660-8b38-4f55feabecfc None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:25.244114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78842f2-0381-4660-8b38-4f55feabecfc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:25.245482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78842f2-0381-4660-8b38-4f55feabecfc None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:25.261546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 647/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 9 02:31:25 2022] POST /identity/v3/services => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:31:27.229964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9082e3-ba05-4e1d-8111-b00b994f5c9b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:27.230953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9082e3-ba05-4e1d-8111-b00b994f5c9b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:27.230953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9082e3-ba05-4e1d-8111-b00b994f5c9b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:27.233060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 649/1296] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:27.238570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:27.239011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:27.239105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:27.259026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:27.260222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:27.335129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:27.358472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9082e3-ba05-4e1d-8111-b00b994f5c9b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:27.470537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9082e3-ba05-4e1d-8111-b00b994f5c9b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:27.505545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:27.506094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:27.506394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:27.506660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:27.507231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:27.508590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:27.517420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:27.518571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:27.519439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 650/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:31:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:27.524930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:27.543543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:27.638976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e05cb5-cae4-4747-ab98-a14c26d66702 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:27.681605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:27.682261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:27.682540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:27.682811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:27.683475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:27.683778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:27.685161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:27.690965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 649/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 9 02:31:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:27.700055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:27.735335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:27.812115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c8ef9-4408-4655-90dc-d7ddac74e395 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:27.859438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:27.860043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:27.860323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:27.860595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:27.861312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] RBAC: Adding query filter params (interface=public, service_id=05a9592ccd4e42b6a29790edab14b643, region_id=RegionOne) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:27.861621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:27.863017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adeffb-15a1-4e06-bf5f-8f515b85d5d1 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:27.868217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 651/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 9 02:31:27 2022] GET /identity/v3/endpoints?service_id=05a9592ccd4e42b6a29790edab14b643&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:27.876748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:27.972217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:27.995660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7459a-8e32-4e2c-a0a8-3111790bf282 None admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:28.030522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648751-b68b-42b6-ba7a-3ca39d5b6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:28.031155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648751-b68b-42b6-ba7a-3ca39d5b6e28 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:28.031439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648751-b68b-42b6-ba7a-3ca39d5b6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:28.031709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648751-b68b-42b6-ba7a-3ca39d5b6e28 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:28.032320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95648751-b68b-42b6-ba7a-3ca39d5b6e28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:28.033690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95648751-b68b-42b6-ba7a-3ca39d5b6e28 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:28.040539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 650/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 9 02:31:27 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:31:30.008199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85a11-331f-4e33-8eb8-0e2f421e4bef None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:30.009208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85a11-331f-4e33-8eb8-0e2f421e4bef None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:30.009208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85a11-331f-4e33-8eb8-0e2f421e4bef None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:30.011331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 652/1302] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:31:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:31:30.016681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:30.017128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:30.017229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:30.036668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:31:30.037825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:31:30.105330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 651/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:31:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 02:31:30.127910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85a11-331f-4e33-8eb8-0e2f421e4bef None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:30.239855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:30.247545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85a11-331f-4e33-8eb8-0e2f421e4bef None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:30.281849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:30.282502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:30.282775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:30.283061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:30.283675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:30.285030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 02:31:30.293864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 02:31:30.295154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application with Lock( Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self._enter() Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application return self.creator() Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application created_value = creator( Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: ERROR keystone.server.flask.application Aug 09 02:31:30.296307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 653/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 9 02:31:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 02:31:30.303994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:30.389256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:30.414501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4275ce2-0fb2-4fcc-9577-7280e76e7dd8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:30.452499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:30.453139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:30.453414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:30.453682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] PATH_INFO: `/v3/services` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:30.454339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:31:30.454641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:30.456003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b369c5-740a-4ea6-99dc-0859c2a33381 None admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:30.461180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 652/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 9 02:31:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:31:30.467512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:31:30.573803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d5a57-588e-41c5-9f1a-29eaebe1dea4 None admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:31:30.611808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce2cfe-0c44-49b3-91f8-d818d9e4300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:31:30.612406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce2cfe-0c44-49b3-91f8-d818d9e4300f None admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:31:30.612693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce2cfe-0c44-49b3-91f8-d818d9e4300f None admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:31:30.612961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce2cfe-0c44-49b3-91f8-d818d9e4300f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:31:30.613623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce2cfe-0c44-49b3-91f8-d818d9e4300f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:31:30.615029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce2cfe-0c44-49b3-91f8-d818d9e4300f None admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:31:30.636541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 654/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:31:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:31:40.458108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=30 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:31:40.632044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:33.478870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459a510-491c-4ecf-af7c-8bc46665beee None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:33.479475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459a510-491c-4ecf-af7c-8bc46665beee None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:33.479475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459a510-491c-4ecf-af7c-8bc46665beee None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:33.481036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:33.486637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b70d-98d0-43c3-be87-a56a994a9e84 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:33.487207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b70d-98d0-43c3-be87-a56a994a9e84 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:33.487207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b70d-98d0-43c3-be87-a56a994a9e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:33.640885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-16f2b70d-98d0-43c3-be87-a56a994a9e84 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:33.642120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-16f2b70d-98d0-43c3-be87-a56a994a9e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:33.845751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:33.905803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d01a8-f617-4cb7-9a99-a28999423c83 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:33.906086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d01a8-f617-4cb7-9a99-a28999423c83 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:33.906389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d01a8-f617-4cb7-9a99-a28999423c83 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:33.907816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 654/1309] 10.222.0.20 () {54 vars in 965 bytes} [Tue Aug 9 02:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:33.928244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562608-913b-489f-a2f9-4ad346f7cd0f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:33.928537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562608-913b-489f-a2f9-4ad346f7cd0f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:33.928900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562608-913b-489f-a2f9-4ad346f7cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:34.082713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-27562608-913b-489f-a2f9-4ad346f7cd0f None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:34.084145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-27562608-913b-489f-a2f9-4ad346f7cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:34.128668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:34.223176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 656/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Aug 9 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:34.223985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d01a8-f617-4cb7-9a99-a28999423c83 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:34.225760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fd6d01a8-f617-4cb7-9a99-a28999423c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:34.374834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d01a8-f617-4cb7-9a99-a28999423c83 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:34.451880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:34.452583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:34.452846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:34.453114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:34.454761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:34.512688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:34.571436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:34.573078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542118d9-723f-4f3d-994d-3aa2f6cc51b3 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:34.593139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:32:36.919285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ba604-fdea-4f97-8951-d9008d6fc089 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:36.919896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ba604-fdea-4f97-8951-d9008d6fc089 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:36.919896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ba604-fdea-4f97-8951-d9008d6fc089 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:36.921182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 657/1312] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:36.926600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5bdfc-0771-4741-892c-ad23847e954e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:36.927060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5bdfc-0771-4741-892c-ad23847e954e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:36.927337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5bdfc-0771-4741-892c-ad23847e954e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:37.021482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-23d5bdfc-0771-4741-892c-ad23847e954e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:37.022765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-23d5bdfc-0771-4741-892c-ad23847e954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:37.127047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:37.165537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ba604-fdea-4f97-8951-d9008d6fc089 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:37.167205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-779ba604-fdea-4f97-8951-d9008d6fc089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:37.277659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ba604-fdea-4f97-8951-d9008d6fc089 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:37.351475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:37.356247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:37.356884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:37.357390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:37.358201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:37.360010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:37.466720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:37.468176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edd8ad-8930-4826-ad14-293bcd4eb27b service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:37.488073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 658/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:32:39.785366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b949d42-362d-4a56-889b-d8f77dbd782c None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:39.786210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b949d42-362d-4a56-889b-d8f77dbd782c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:39.786210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b949d42-362d-4a56-889b-d8f77dbd782c None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:39.788394 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 657/1315] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:39.794744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e21b6-36c7-4825-ad72-e5a5b50686bf None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:39.795476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e21b6-36c7-4825-ad72-e5a5b50686bf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:39.795634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e21b6-36c7-4825-ad72-e5a5b50686bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:39.871343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e14e21b6-36c7-4825-ad72-e5a5b50686bf None None] MFA Rules not processed for user `186bbe5bc1074e8890ca95419df7673e`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:39.872613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e14e21b6-36c7-4825-ad72-e5a5b50686bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:39.934923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:39.979718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 659/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:40.025168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b949d42-362d-4a56-889b-d8f77dbd782c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:40.027712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3b949d42-362d-4a56-889b-d8f77dbd782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:40.042944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:40.141944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b949d42-362d-4a56-889b-d8f77dbd782c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:40.215368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:40.215971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:40.216247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:40.216516 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:40.218113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:40.281817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:40.333387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:40.334881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afd221-1f88-4633-994b-209989931c2f service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:40.353356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 658/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:32:42.948571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da956cc0-b16b-454e-9393-a688f22ad45f None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:42.949222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da956cc0-b16b-454e-9393-a688f22ad45f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:42.949222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da956cc0-b16b-454e-9393-a688f22ad45f None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:42.950580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 660/1318] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:42.956971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd208c7-af78-449a-8cce-e277c36ec09a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:42.957710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd208c7-af78-449a-8cce-e277c36ec09a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:42.957914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd208c7-af78-449a-8cce-e277c36ec09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:43.041533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1dd208c7-af78-449a-8cce-e277c36ec09a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:43.042874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1dd208c7-af78-449a-8cce-e277c36ec09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:43.146530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 659/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:43.203494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597052b1-0baa-48fa-b271-0c490e294512 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:43.203699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597052b1-0baa-48fa-b271-0c490e294512 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:43.204077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597052b1-0baa-48fa-b271-0c490e294512 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:43.206295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 661/1320] 10.222.0.20 () {54 vars in 965 bytes} [Tue Aug 9 02:32:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:43.213038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c1ab-4793-4481-aa49-dd8b57817bdb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:43.213405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c1ab-4793-4481-aa49-dd8b57817bdb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:43.213792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c1ab-4793-4481-aa49-dd8b57817bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:43.237086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:43.295568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1b61c1ab-4793-4481-aa49-dd8b57817bdb None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:43.296792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1b61c1ab-4793-4481-aa49-dd8b57817bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:43.387674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 660/1321] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Aug 9 02:32:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:43.396951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597052b1-0baa-48fa-b271-0c490e294512 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:43.398546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-597052b1-0baa-48fa-b271-0c490e294512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:43.515220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597052b1-0baa-48fa-b271-0c490e294512 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:43.595231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:43.595853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:43.596132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:43.596384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:43.598015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:43.605966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:43.710203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:43.711673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d005a5-3840-41c3-8861-57d23be8797e service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:43.730132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 662/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:32:46.844711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380a23-3ac6-45a0-8322-26c965ba7bba None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:46.845343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380a23-3ac6-45a0-8322-26c965ba7bba None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:46.845343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380a23-3ac6-45a0-8322-26c965ba7bba None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:46.846682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 661/1323] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:46.852411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584b9d0-9880-4635-a70f-1718787f9c8f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:46.852855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584b9d0-9880-4635-a70f-1718787f9c8f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:46.852966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584b9d0-9880-4635-a70f-1718787f9c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:46.921153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e584b9d0-9880-4635-a70f-1718787f9c8f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:46.922510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e584b9d0-9880-4635-a70f-1718787f9c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:46.986274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:47.030448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 663/1324] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:47.067153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380a23-3ac6-45a0-8322-26c965ba7bba None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:47.068826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cf380a23-3ac6-45a0-8322-26c965ba7bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:47.182311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380a23-3ac6-45a0-8322-26c965ba7bba None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:47.195170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:47.260158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:47.260813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:47.261094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:47.261364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:47.263053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:47.381565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:47.382948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291ea29-850b-46f0-9e91-5589bacb671e service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:47.401670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 662/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:32:49.448001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7116bfe-d3ac-44ef-8096-c7e77f685ed7 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:49.448535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7116bfe-d3ac-44ef-8096-c7e77f685ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:49.448535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7116bfe-d3ac-44ef-8096-c7e77f685ed7 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:49.449904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 664/1326] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:49.455183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623e107-e08e-46e5-9d8d-61541f7f5857 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:49.455602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623e107-e08e-46e5-9d8d-61541f7f5857 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:49.455777 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623e107-e08e-46e5-9d8d-61541f7f5857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:49.506924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:49.526008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d623e107-e08e-46e5-9d8d-61541f7f5857 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:49.527253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d623e107-e08e-46e5-9d8d-61541f7f5857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:49.628527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 663/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:49.664398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7116bfe-d3ac-44ef-8096-c7e77f685ed7 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:49.665991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c7116bfe-d3ac-44ef-8096-c7e77f685ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:49.770730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7116bfe-d3ac-44ef-8096-c7e77f685ed7 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:49.848212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:49.848837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:49.849114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:49.849385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:49.851018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:49.869464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:49.969354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:49.970843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a76536-37a0-4a4a-854a-8218cf9beec0 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:49.990000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 665/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:32:52.037293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc17f0-f46d-4169-aa8b-a2b929d6dae8 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:52.037889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc17f0-f46d-4169-aa8b-a2b929d6dae8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:52.037889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc17f0-f46d-4169-aa8b-a2b929d6dae8 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:52.039246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 664/1329] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:52.044771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd9836-b38c-4854-afd1-75345c66db9f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:52.045216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd9836-b38c-4854-afd1-75345c66db9f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:52.045307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd9836-b38c-4854-afd1-75345c66db9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:52.111703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6ecd9836-b38c-4854-afd1-75345c66db9f None None] MFA Rules not processed for user `186bbe5bc1074e8890ca95419df7673e`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:52.112915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6ecd9836-b38c-4854-afd1-75345c66db9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:52.178803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:52.212462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 666/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:52.250509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc17f0-f46d-4169-aa8b-a2b929d6dae8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:52.252103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-57fc17f0-f46d-4169-aa8b-a2b929d6dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:52.364831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc17f0-f46d-4169-aa8b-a2b929d6dae8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:52.385679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:52.443090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:52.443623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:52.443930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:52.444193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:52.445785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:52.557374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:52.558908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c49b03-57e9-4456-bef5-ab9eee8c3d22 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:52.577938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 665/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:32:54.797171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766ba8-b87e-47d9-bdb6-dc1d707c2c62 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:54.797630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766ba8-b87e-47d9-bdb6-dc1d707c2c62 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:54.797714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766ba8-b87e-47d9-bdb6-dc1d707c2c62 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:54.799140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 667/1332] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:54.805619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57350bee-62c9-4ce9-bb01-de492b7b8947 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:54.806066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57350bee-62c9-4ce9-bb01-de492b7b8947 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:54.806204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57350bee-62c9-4ce9-bb01-de492b7b8947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:54.870187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:54.873214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-57350bee-62c9-4ce9-bb01-de492b7b8947 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:54.875082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-57350bee-62c9-4ce9-bb01-de492b7b8947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:54.998047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 666/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:55.035476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766ba8-b87e-47d9-bdb6-dc1d707c2c62 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:55.037059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-67766ba8-b87e-47d9-bdb6-dc1d707c2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:55.139615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766ba8-b87e-47d9-bdb6-dc1d707c2c62 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:55.212940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:55.213543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:55.213823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:55.214099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:55.215832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:55.244331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:55.331722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:55.333161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf6be6-d075-4fa0-9333-dc5e28dbd644 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:55.352873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 668/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:32:57.391986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe47563-d9f9-45ff-aadd-9f373172f64d None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:57.392721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe47563-d9f9-45ff-aadd-9f373172f64d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:57.392815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe47563-d9f9-45ff-aadd-9f373172f64d None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:57.394965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 667/1335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:32:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:32:57.400336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78ef6f-1670-40a3-94cb-9075d8deb059 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:57.400812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78ef6f-1670-40a3-94cb-9075d8deb059 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:57.400939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78ef6f-1670-40a3-94cb-9075d8deb059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:57.469325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6c78ef6f-1670-40a3-94cb-9075d8deb059 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:32:57.470564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6c78ef6f-1670-40a3-94cb-9075d8deb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:57.546391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:57.570415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 669/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:32:57.613221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe47563-d9f9-45ff-aadd-9f373172f64d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:32:57.614829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fbe47563-d9f9-45ff-aadd-9f373172f64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:57.733876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe47563-d9f9-45ff-aadd-9f373172f64d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:32:57.767847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:32:57.814670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:32:57.815268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:32:57.815547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:32:57.815826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:32:57.817417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:32:57.932751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:32:57.934265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac07eb0-b906-40b2-abb8-7edf973c7f75 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:32:57.953034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 668/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:00.369704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc847-322b-4804-8c15-30752dd0c283 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:00.370312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc847-322b-4804-8c15-30752dd0c283 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:00.370312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc847-322b-4804-8c15-30752dd0c283 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:00.371664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 670/1338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:00.377973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213210-940c-431f-9abe-db188a37edbb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:00.378810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213210-940c-431f-9abe-db188a37edbb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:00.378946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213210-940c-431f-9abe-db188a37edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:00.453726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b5213210-940c-431f-9abe-db188a37edbb None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:00.454978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b5213210-940c-431f-9abe-db188a37edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:00.463753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:00.566455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 669/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:00.603261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bc847-322b-4804-8c15-30752dd0c283 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:00.604926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-913bc847-322b-4804-8c15-30752dd0c283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:00.710716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bc847-322b-4804-8c15-30752dd0c283 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:00.789158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:00.789777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:00.790054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:00.790357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:00.791960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:00.826708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:00.908166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:00.909605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3b42b-d3c6-43bf-986a-f860400abdbd service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:00.931301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 671/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:05.098867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754079e-e3d2-41cb-85b9-243f88b345e7 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:05.099455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754079e-e3d2-41cb-85b9-243f88b345e7 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:05.099455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754079e-e3d2-41cb-85b9-243f88b345e7 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:05.100721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 670/1341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:05.106578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c61ad-5928-4360-89f1-45b91d7ceb72 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:05.107051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c61ad-5928-4360-89f1-45b91d7ceb72 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:05.107162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c61ad-5928-4360-89f1-45b91d7ceb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:05.177622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1f6c61ad-5928-4360-89f1-45b91d7ceb72 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:05.178815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1f6c61ad-5928-4360-89f1-45b91d7ceb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:05.266716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:05.278986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 672/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:05.318105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754079e-e3d2-41cb-85b9-243f88b345e7 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:05.319827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b754079e-e3d2-41cb-85b9-243f88b345e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:05.459124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754079e-e3d2-41cb-85b9-243f88b345e7 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:05.497003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:05.533082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:05.533686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:05.533960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:05.534254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:05.535836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:05.642792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:05.644143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e801-026c-4159-a0f9-ee5b7ce83fa6 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:05.662487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 671/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:08.235812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00196d-d083-4b0a-9734-714b1b299757 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:08.236223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00196d-d083-4b0a-9734-714b1b299757 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:08.236400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00196d-d083-4b0a-9734-714b1b299757 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:08.237770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 673/1344] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:08.243927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db153f-d0bb-4dee-829a-104db6afeebb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:08.244403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db153f-d0bb-4dee-829a-104db6afeebb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:08.244502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db153f-d0bb-4dee-829a-104db6afeebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:08.314251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b0db153f-d0bb-4dee-829a-104db6afeebb None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:08.315502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b0db153f-d0bb-4dee-829a-104db6afeebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:08.334685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:08.419215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 672/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:08.456859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00196d-d083-4b0a-9734-714b1b299757 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:08.458474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-df00196d-d083-4b0a-9734-714b1b299757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:08.565679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00196d-d083-4b0a-9734-714b1b299757 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:08.641576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408c821-9612-48be-8380-86e009352141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:08.642188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c821-9612-48be-8380-86e009352141 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:08.642492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c821-9612-48be-8380-86e009352141 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:08.642774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c821-9612-48be-8380-86e009352141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:08.644337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f408c821-9612-48be-8380-86e009352141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:08.688376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:08.754397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408c821-9612-48be-8380-86e009352141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:08.755839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408c821-9612-48be-8380-86e009352141 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:08.774832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 674/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:13.088299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98492fe-2304-4acb-b96f-b366fc0a5304 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:13.088831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98492fe-2304-4acb-b96f-b366fc0a5304 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:13.088831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98492fe-2304-4acb-b96f-b366fc0a5304 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:13.090242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 673/1347] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:13.095709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2dd6-1e72-4420-8e4b-e48dfa065d20 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:13.096203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2dd6-1e72-4420-8e4b-e48dfa065d20 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:13.096203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2dd6-1e72-4420-8e4b-e48dfa065d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:13.175462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8f9a2dd6-1e72-4420-8e4b-e48dfa065d20 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:13.176697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8f9a2dd6-1e72-4420-8e4b-e48dfa065d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:13.280020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:13.282665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 675/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:13.320388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98492fe-2304-4acb-b96f-b366fc0a5304 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:13.321959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d98492fe-2304-4acb-b96f-b366fc0a5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:13.448688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98492fe-2304-4acb-b96f-b366fc0a5304 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:13.515568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:13.539581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:13.539731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:13.540012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:13.540284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:13.541941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:13.660483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:13.662001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f362280-40a8-4f29-bcf6-9806a9cd91b0 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:13.680576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 674/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:18.523857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ffa5-0e54-4d7b-940f-167827a35078 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:18.524354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ffa5-0e54-4d7b-940f-167827a35078 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:18.524555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ffa5-0e54-4d7b-940f-167827a35078 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:18.526190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 676/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:18.531955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c96de-1989-403c-98c2-689d2626c85e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:18.532385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c96de-1989-403c-98c2-689d2626c85e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:18.533354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c96de-1989-403c-98c2-689d2626c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:18.622705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-696c96de-1989-403c-98c2-689d2626c85e None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:18.623896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-696c96de-1989-403c-98c2-689d2626c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:18.642088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:18.738115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 675/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:33:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:22.944060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93fec7-7c70-4ac7-8e62-da92a3479483 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:22.944680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93fec7-7c70-4ac7-8e62-da92a3479483 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:22.944680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93fec7-7c70-4ac7-8e62-da92a3479483 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:22.946132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 677/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:22.951239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f14b18-7ba2-4475-8566-601a019d5333 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:22.951709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f14b18-7ba2-4475-8566-601a019d5333 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:22.951793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f14b18-7ba2-4475-8566-601a019d5333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:23.040062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a1f14b18-7ba2-4475-8566-601a019d5333 None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:23.041612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a1f14b18-7ba2-4475-8566-601a019d5333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:23.117894 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:23.157145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 676/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:33:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:24.193604 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dddb3f-ced6-419f-8ece-211320d66a1e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:24.194244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dddb3f-ced6-419f-8ece-211320d66a1e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:24.194244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dddb3f-ced6-419f-8ece-211320d66a1e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:24.195494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 678/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:33:24.200663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609dda3-f158-4f0d-abd1-a071cb097cb8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:24.201094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609dda3-f158-4f0d-abd1-a071cb097cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:24.201243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609dda3-f158-4f0d-abd1-a071cb097cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:24.279915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f609dda3-f158-4f0d-abd1-a071cb097cb8 None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:24.281283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f609dda3-f158-4f0d-abd1-a071cb097cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:24.395764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 677/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:33:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:27.753251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc697b53-ce4c-4c72-a4e9-21683297728d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:27.753790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc697b53-ce4c-4c72-a4e9-21683297728d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:27.753790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc697b53-ce4c-4c72-a4e9-21683297728d None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:27.755390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:27.761067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c018b5-6f45-45f3-9e2b-7812c7664b87 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:27.761549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c018b5-6f45-45f3-9e2b-7812c7664b87 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:27.761672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c018b5-6f45-45f3-9e2b-7812c7664b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:27.807040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:27.853692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-28c018b5-6f45-45f3-9e2b-7812c7664b87 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:27.855792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-28c018b5-6f45-45f3-9e2b-7812c7664b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:27.966276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 678/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:28.020983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347816dc-2763-438c-84b2-3367d5813552 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:28.021647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347816dc-2763-438c-84b2-3367d5813552 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:28.022216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347816dc-2763-438c-84b2-3367d5813552 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:28.023867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 680/1358] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 9 02:33:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:28.032447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479dff-32b7-4090-aa08-bd2ce71a79dc None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:28.032855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479dff-32b7-4090-aa08-bd2ce71a79dc None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:28.033552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479dff-32b7-4090-aa08-bd2ce71a79dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:28.119435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5e479dff-32b7-4090-aa08-bd2ce71a79dc None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:28.120669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5e479dff-32b7-4090-aa08-bd2ce71a79dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:28.173803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:28.244042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 679/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 9 02:33:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:28.251990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347816dc-2763-438c-84b2-3367d5813552 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:28.253489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-347816dc-2763-438c-84b2-3367d5813552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:28.398710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347816dc-2763-438c-84b2-3367d5813552 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:28.477075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:28.517595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:28.518311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:28.518701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:28.519053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:28.520854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:28.611370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:28.613113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64022a9a-c40e-407e-b1f9-74cef97b0c6f service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:28.658539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 681/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:30.852447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdcfcb-b15b-451c-a546-8f1086346018 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:30.853022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdcfcb-b15b-451c-a546-8f1086346018 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:30.853022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdcfcb-b15b-451c-a546-8f1086346018 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:30.854811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 680/1361] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:30.865285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c94f6-3e38-4a26-a2f5-125c56b1ade3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:30.865726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c94f6-3e38-4a26-a2f5-125c56b1ade3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:30.865966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c94f6-3e38-4a26-a2f5-125c56b1ade3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:30.944768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-396c94f6-3e38-4a26-a2f5-125c56b1ade3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:30.946414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-396c94f6-3e38-4a26-a2f5-125c56b1ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:30.989560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:31.058193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:31.100193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdcfcb-b15b-451c-a546-8f1086346018 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:31.102204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-52cdcfcb-b15b-451c-a546-8f1086346018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:31.292312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdcfcb-b15b-451c-a546-8f1086346018 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:31.303341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:31.439214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:31.440039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:31.440391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:31.440732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:31.442776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:31.542622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:31.544248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de57da-e568-4c8c-84b3-30cbb2b08b34 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:31.563642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 681/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:33.921040 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269199e7-46df-4e62-a843-903d71ad83d5 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:33.921654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269199e7-46df-4e62-a843-903d71ad83d5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:33.921775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269199e7-46df-4e62-a843-903d71ad83d5 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:33.923377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 683/1364] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:33.928682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427c84-84af-4134-a6e6-cbe476ed2cb3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:33.929502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427c84-84af-4134-a6e6-cbe476ed2cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:33.929795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427c84-84af-4134-a6e6-cbe476ed2cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:33.952937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:34.009631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f7427c84-84af-4134-a6e6-cbe476ed2cb3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:34.011096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f7427c84-84af-4134-a6e6-cbe476ed2cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:34.122712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 682/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:34.161912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3b87-c804-42bd-84f9-1d35b2f21597 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:34.162373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3b87-c804-42bd-84f9-1d35b2f21597 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:34.162902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3b87-c804-42bd-84f9-1d35b2f21597 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:34.164444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 684/1366] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 9 02:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:34.170519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447854f-3d4b-4d6b-a570-3c118581ce73 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:34.170815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447854f-3d4b-4d6b-a570-3c118581ce73 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:34.171421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447854f-3d4b-4d6b-a570-3c118581ce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:34.265097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e447854f-3d4b-4d6b-a570-3c118581ce73 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:34.266411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e447854f-3d4b-4d6b-a570-3c118581ce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:34.303436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:34.391697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 683/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 9 02:33:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:34.399719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3b87-c804-42bd-84f9-1d35b2f21597 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:34.401333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5dfb3b87-c804-42bd-84f9-1d35b2f21597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:34.552693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3b87-c804-42bd-84f9-1d35b2f21597 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:34.574181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:34.672507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:34.673230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:34.673579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:34.673952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:34.675684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:34.769704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:34.771487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457b603-883a-49dd-9369-2a6966ccde24 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:34.790185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 685/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:35.761335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897061de-b4aa-4466-86f4-f5f30a61bf86 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:35.761920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897061de-b4aa-4466-86f4-f5f30a61bf86 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:35.761920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897061de-b4aa-4466-86f4-f5f30a61bf86 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:35.763520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 684/1369] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:35.769285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6a16-79bd-4504-8a35-00196d77154f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:35.769779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6a16-79bd-4504-8a35-00196d77154f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:35.769779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6a16-79bd-4504-8a35-00196d77154f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:35.802385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:35.837281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f83e6a16-79bd-4504-8a35-00196d77154f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:35.838637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f83e6a16-79bd-4504-8a35-00196d77154f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:35.947114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 686/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:36.282815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16522-4c8c-49b2-8460-dc47eb894ba4 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:36.283101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16522-4c8c-49b2-8460-dc47eb894ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:36.283377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16522-4c8c-49b2-8460-dc47eb894ba4 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:36.284886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 685/1371] 10.222.0.20 () {54 vars in 964 bytes} [Tue Aug 9 02:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:36.289652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f93ec7-68ea-416a-ae7a-d7a513ccb8f7 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:36.290006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f93ec7-68ea-416a-ae7a-d7a513ccb8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:36.290267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f93ec7-68ea-416a-ae7a-d7a513ccb8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:36.368940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b1f93ec7-68ea-416a-ae7a-d7a513ccb8f7 None None] MFA Rules not processed for user `d57751a2455c48d28fcf9e97b62cfc38`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:36.370246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b1f93ec7-68ea-416a-ae7a-d7a513ccb8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:36.426570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:36.461500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 687/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Aug 9 02:33:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:36.469685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16522-4c8c-49b2-8460-dc47eb894ba4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:36.471266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-aba16522-4c8c-49b2-8460-dc47eb894ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:36.582558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16522-4c8c-49b2-8460-dc47eb894ba4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:36.624217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:36.659936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:36.660544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:36.660825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:36.661099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:36.662767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:36.785138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:36.787448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47024c-4973-407f-b664-03b1db723d9f service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:36.808831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 686/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:36.896314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e9cb0-8221-438b-a726-80850c36ee95 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:36.896771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e9cb0-8221-438b-a726-80850c36ee95 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:36.897019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e9cb0-8221-438b-a726-80850c36ee95 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:36.899008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 688/1374] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:36.904427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b508db8-33ae-4109-9050-5104c4c2de4c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:36.904721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b508db8-33ae-4109-9050-5104c4c2de4c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:36.904996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b508db8-33ae-4109-9050-5104c4c2de4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:36.972508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6b508db8-33ae-4109-9050-5104c4c2de4c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:36.973843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6b508db8-33ae-4109-9050-5104c4c2de4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:36.995673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:37.078696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 687/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:37.119724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e9cb0-8221-438b-a726-80850c36ee95 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:37.121148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4e3e9cb0-8221-438b-a726-80850c36ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:37.285414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e9cb0-8221-438b-a726-80850c36ee95 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:37.339795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:37.405413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:37.406043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:37.406349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:37.406620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:37.408265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:37.503046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:37.504642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e91be-1961-43c7-a882-a68585da5fa0 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:37.523020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 689/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:39.527075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7097560-92d9-4352-8fa4-45251cfb4fc0 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:39.527595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7097560-92d9-4352-8fa4-45251cfb4fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:39.527719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7097560-92d9-4352-8fa4-45251cfb4fc0 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:39.529086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 688/1377] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:39.534760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297cbd0-90c7-4aaf-92a0-8c3c4df2db93 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:39.535234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297cbd0-90c7-4aaf-92a0-8c3c4df2db93 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:39.535333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297cbd0-90c7-4aaf-92a0-8c3c4df2db93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:39.602219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8297cbd0-90c7-4aaf-92a0-8c3c4df2db93 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:39.603434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8297cbd0-90c7-4aaf-92a0-8c3c4df2db93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:39.612323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:39.709798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 690/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:39.760719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7097560-92d9-4352-8fa4-45251cfb4fc0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:39.762506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b7097560-92d9-4352-8fa4-45251cfb4fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:39.933186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7097560-92d9-4352-8fa4-45251cfb4fc0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:39.937083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:40.056701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:40.057425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:40.057692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:40.057961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:40.059600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:40.168077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:40.169951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4618b80-04f9-41a1-8968-29f6823ab6cf service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:40.194581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 689/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:42.252933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad70953-0893-4869-ad58-0fade116e5bf None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:42.253555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad70953-0893-4869-ad58-0fade116e5bf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:42.253555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad70953-0893-4869-ad58-0fade116e5bf None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:42.255123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 691/1380] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:42.261450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039abe7-ce8d-4a7e-b343-5627cc1bef7a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:42.262066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039abe7-ce8d-4a7e-b343-5627cc1bef7a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:42.262066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039abe7-ce8d-4a7e-b343-5627cc1bef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:42.275798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:42.336369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6039abe7-ce8d-4a7e-b343-5627cc1bef7a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:42.337597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6039abe7-ce8d-4a7e-b343-5627cc1bef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:42.443576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 690/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:42.487914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad70953-0893-4869-ad58-0fade116e5bf None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:42.489476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8ad70953-0893-4869-ad58-0fade116e5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:42.642970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:42.650660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad70953-0893-4869-ad58-0fade116e5bf None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:42.775659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:42.776279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:42.776563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:42.776838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:42.778457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:42.876263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:42.877855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd734aa-fa6c-4364-9da2-c3c81db67d83 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:42.888716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:42.897914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 692/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:44.968099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19213-03d9-4eb1-ad3c-7fdd947c5e2f None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:44.968692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19213-03d9-4eb1-ad3c-7fdd947c5e2f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:44.968692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19213-03d9-4eb1-ad3c-7fdd947c5e2f None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:44.970835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 691/1383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:44.976720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd4054-4c4d-4f0b-8835-5c236558548a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:44.977191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd4054-4c4d-4f0b-8835-5c236558548a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:44.977370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd4054-4c4d-4f0b-8835-5c236558548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:45.045566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0fbd4054-4c4d-4f0b-8835-5c236558548a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:45.046863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0fbd4054-4c4d-4f0b-8835-5c236558548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:45.144962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 693/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:45.190562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19213-03d9-4eb1-ad3c-7fdd947c5e2f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:45.192202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-09b19213-03d9-4eb1-ad3c-7fdd947c5e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:45.285778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:45.367447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19213-03d9-4eb1-ad3c-7fdd947c5e2f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:45.496647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:45.497325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:45.497624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:45.497904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:45.499607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:45.552686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:45.595567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:45.598124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c510d-a5b8-4a20-ad4c-5e1d275adf4d service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:45.622773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 692/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:47.638681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99335-55d5-4277-9a86-38f7aee82e1d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:47.639778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99335-55d5-4277-9a86-38f7aee82e1d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:47.640214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99335-55d5-4277-9a86-38f7aee82e1d None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:47.642138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 694/1386] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:47.647625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453a925-0042-4d2b-a8e9-338864bffa01 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:47.648093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453a925-0042-4d2b-a8e9-338864bffa01 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:47.648191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453a925-0042-4d2b-a8e9-338864bffa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:47.718868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5453a925-0042-4d2b-a8e9-338864bffa01 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:47.720047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5453a925-0042-4d2b-a8e9-338864bffa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:47.818818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 693/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:47.878481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99335-55d5-4277-9a86-38f7aee82e1d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:47.880195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-45c99335-55d5-4277-9a86-38f7aee82e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:47.967008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:48.070439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99335-55d5-4277-9a86-38f7aee82e1d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:48.211236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:48.211874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:48.212151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:48.212524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:48.214261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:48.256570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:48.322947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:48.326417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baecf9-e073-4de6-b5ea-87c3b85dbc4e service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:48.354189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 695/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:50.415083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb1d39-bdc9-41f6-8b72-20f040e40f1a None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:50.415583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb1d39-bdc9-41f6-8b72-20f040e40f1a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:50.415583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb1d39-bdc9-41f6-8b72-20f040e40f1a None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:50.417117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 694/1389] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:50.422558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef987b-1142-47c1-ab1f-1dce4e053233 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:50.422996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef987b-1142-47c1-ab1f-1dce4e053233 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:50.423164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef987b-1142-47c1-ab1f-1dce4e053233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:50.494880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c3ef987b-1142-47c1-ab1f-1dce4e053233 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:50.496109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c3ef987b-1142-47c1-ab1f-1dce4e053233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:50.597091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 696/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:50.642194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb1d39-bdc9-41f6-8b72-20f040e40f1a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:50.643751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cfbb1d39-bdc9-41f6-8b72-20f040e40f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:50.675421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:50.793280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb1d39-bdc9-41f6-8b72-20f040e40f1a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:50.901419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:50.913692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:50.914347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:50.914624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:50.914890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:50.916493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:51.133659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:51.135363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a94691-2bb9-4e41-aa1f-8421d4e2c513 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:51.153925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 695/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:51.732924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9291a7-271a-43bd-80e0-c08c6737f474 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:51.733483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9291a7-271a-43bd-80e0-c08c6737f474 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:51.733483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9291a7-271a-43bd-80e0-c08c6737f474 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:51.734953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 697/1392] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:51.740182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de87f9-aa60-4fe2-9a34-95d3fd7478e2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:51.740671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de87f9-aa60-4fe2-9a34-95d3fd7478e2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:51.740764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de87f9-aa60-4fe2-9a34-95d3fd7478e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:51.807994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-18de87f9-aa60-4fe2-9a34-95d3fd7478e2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:51.809226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-18de87f9-aa60-4fe2-9a34-95d3fd7478e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:51.871754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:51.910308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 696/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:51.966443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9e803-ee79-4547-9de6-b92c7642bc7b None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:51.966733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9e803-ee79-4547-9de6-b92c7642bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:51.966982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9e803-ee79-4547-9de6-b92c7642bc7b None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:51.968359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 698/1394] 10.222.0.20 () {54 vars in 967 bytes} [Tue Aug 9 02:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:51.973452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26bfab-fd28-4e53-b83b-1c0ba1bf8b7f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:51.973740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26bfab-fd28-4e53-b83b-1c0ba1bf8b7f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:51.974008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26bfab-fd28-4e53-b83b-1c0ba1bf8b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:52.053539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1a26bfab-fd28-4e53-b83b-1c0ba1bf8b7f None None] MFA Rules not processed for user `7a0b055ed0d14d3e9428d1291f0f1209`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:52.054719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1a26bfab-fd28-4e53-b83b-1c0ba1bf8b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:52.133960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 697/1395] 10.222.0.20 () {58 vars in 1064 bytes} [Tue Aug 9 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:52.141967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9e803-ee79-4547-9de6-b92c7642bc7b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:52.143629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-74f9e803-ee79-4547-9de6-b92c7642bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:52.157711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:52.250620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9e803-ee79-4547-9de6-b92c7642bc7b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:52.336340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:52.336964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:52.337245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:52.337518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:52.339214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:52.410106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:52.455202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:52.456653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c40f38-88dd-478e-ba45-5b50f1c1ff4d service glance] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:52.476252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 699/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:53.193722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b98bc-d7b0-41de-943b-76e3d4c046bf None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.194337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b98bc-d7b0-41de-943b-76e3d4c046bf None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.194337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b98bc-d7b0-41de-943b-76e3d4c046bf None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.198189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 698/1397] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:53.201273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ead3f-2775-43d4-ad57-441edb25be7a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.201710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ead3f-2775-43d4-ad57-441edb25be7a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.201940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ead3f-2775-43d4-ad57-441edb25be7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.214260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.214562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.214884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.270536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-812ead3f-2775-43d4-ad57-441edb25be7a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:53.271767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-812ead3f-2775-43d4-ad57-441edb25be7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:53.283591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:53.284789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:53.293064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:53.373121 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 700/1398] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:53.378342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 699/1399] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:33:53.418255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190abb2-a814-4743-9145-a6c71a27e898 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.418552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190abb2-a814-4743-9145-a6c71a27e898 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.418825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190abb2-a814-4743-9145-a6c71a27e898 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.419982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:53.420352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 701/1400] 10.222.0.20 () {54 vars in 943 bytes} [Tue Aug 9 02:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:53.421199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:53.426496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.426799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.427077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.452827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:53.503834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] MFA Rules not processed for user `5ea3e582aac94931b1a041817ccab685`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:53.504967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:53.540934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e189ec-9628-4661-9725-a9d1386d997a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:53.582509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:53.613149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 702/1401] 10.222.0.20 () {58 vars in 1040 bytes} [Tue Aug 9 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:53.622600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:53.631865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba344cf-711b-4115-98aa-c4fd67709da1 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:53.658430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:53.659046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.659315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.659583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.661173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:53.701310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:53.736062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:53.736679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:53.736953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:53.737220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:53.738996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:53.755398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:53.756955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d7927-8e5f-4b5a-9d3e-3a37904c3174 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:53.775852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 700/1402] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:53.822036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:53.823650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9a15-f4d1-4e7f-a15e-56570725fea8 service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:53.832068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 703/1403] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:33:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 02:33:55.817923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1472-a49e-46f5-bbef-9c847e7f60e3 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:55.818492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1472-a49e-46f5-bbef-9c847e7f60e3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:55.818492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1472-a49e-46f5-bbef-9c847e7f60e3 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:55.819794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 701/1404] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:55.825172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dc772-0970-4259-a763-4089254f1344 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:55.825679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dc772-0970-4259-a763-4089254f1344 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:55.825779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dc772-0970-4259-a763-4089254f1344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:55.896683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1d7dc772-0970-4259-a763-4089254f1344 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:55.898308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1d7dc772-0970-4259-a763-4089254f1344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:55.958962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:56.008527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 704/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:56.058838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b1472-a49e-46f5-bbef-9c847e7f60e3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:56.060508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8d6b1472-a49e-46f5-bbef-9c847e7f60e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:56.118329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:56.234740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b1472-a49e-46f5-bbef-9c847e7f60e3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:56.364691 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:56.365316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:56.365590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:56.365857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:56.367531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:56.375903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:56.474234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:56.476324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9149e-8bcb-4d04-8948-a825546971e6 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:56.495179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 702/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:33:58.562675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5890d-e4af-4404-9002-1dc8ed36b893 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:58.563349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5890d-e4af-4404-9002-1dc8ed36b893 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:58.563349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5890d-e4af-4404-9002-1dc8ed36b893 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:58.566724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 705/1407] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:33:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:33:58.570878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f81671-cdbe-43db-aedb-b5b59d4ed6cb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:58.571452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f81671-cdbe-43db-aedb-b5b59d4ed6cb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:58.571452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f81671-cdbe-43db-aedb-b5b59d4ed6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:58.654681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-98f81671-cdbe-43db-aedb-b5b59d4ed6cb None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:33:58.655804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-98f81671-cdbe-43db-aedb-b5b59d4ed6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:58.748928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:58.772715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 703/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:33:58.818181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5890d-e4af-4404-9002-1dc8ed36b893 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:33:58.819727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-09f5890d-e4af-4404-9002-1dc8ed36b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:58.975593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5890d-e4af-4404-9002-1dc8ed36b893 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:33:59.024455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:33:59.109184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:33:59.109923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:33:59.110295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:33:59.110565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:33:59.112151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:33:59.218600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:33:59.220231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fefd4a-4b0c-4a6b-8d8f-77f969fb875b service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:33:59.241044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 706/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:08.770748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=7 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:09.238591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=79 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:15.579863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159fcea-8249-45e3-9dab-76799af26d07 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:15.580464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159fcea-8249-45e3-9dab-76799af26d07 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:15.580464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159fcea-8249-45e3-9dab-76799af26d07 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:15.581773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 704/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Aug 9 02:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:15.587193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449ed06-efa7-4221-a7d9-c9825ab348d9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:15.587648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449ed06-efa7-4221-a7d9-c9825ab348d9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:15.587758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449ed06-efa7-4221-a7d9-c9825ab348d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:15.674371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4449ed06-efa7-4221-a7d9-c9825ab348d9 None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:15.675496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4449ed06-efa7-4221-a7d9-c9825ab348d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:15.787299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 707/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 9 02:34:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:15.801943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c862-ed26-440a-b315-8cc4ba0ccae3 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:15.802255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c862-ed26-440a-b315-8cc4ba0ccae3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:15.802527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c862-ed26-440a-b315-8cc4ba0ccae3 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:15.803924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 705/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Aug 9 02:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:15.808765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98441c77-139d-4e6a-bf60-e47a522c9306 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:15.809051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98441c77-139d-4e6a-bf60-e47a522c9306 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:15.809322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98441c77-139d-4e6a-bf60-e47a522c9306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:15.883018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:15.885936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-98441c77-139d-4e6a-bf60-e47a522c9306 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:15.887841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-98441c77-139d-4e6a-bf60-e47a522c9306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:16.022662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 708/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 9 02:34:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:18.500461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934e4a3-c12e-4a11-9724-1f711575d979 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:18.501735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934e4a3-c12e-4a11-9724-1f711575d979 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:18.501735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934e4a3-c12e-4a11-9724-1f711575d979 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:18.503745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:18.509261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ec9d-5f5a-44a5-9c72-c376aeb58888 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:18.509703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ec9d-5f5a-44a5-9c72-c376aeb58888 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:18.510133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ec9d-5f5a-44a5-9c72-c376aeb58888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:18.579515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6164ec9d-5f5a-44a5-9c72-c376aeb58888 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:18.580713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6164ec9d-5f5a-44a5-9c72-c376aeb58888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:18.647769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:18.692505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:18.735327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934e4a3-c12e-4a11-9724-1f711575d979 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:18.736837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d934e4a3-c12e-4a11-9724-1f711575d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:18.891332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934e4a3-c12e-4a11-9724-1f711575d979 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:18.967384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:19.010639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:19.011245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:19.011523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:19.011790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:19.013374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:19.108678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:19.110329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fb5c7-c190-4b13-b41b-84b59b104740 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:19.128936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 707/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:20.039259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec73a06-eb82-4ebc-bfce-4f20c0fb7a87 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:20.039884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec73a06-eb82-4ebc-bfce-4f20c0fb7a87 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:20.039884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec73a06-eb82-4ebc-bfce-4f20c0fb7a87 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:20.041212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:20.047936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62115cc6-5a9f-4f8f-81cc-559fedb0a7ba None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:20.048680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62115cc6-5a9f-4f8f-81cc-559fedb0a7ba None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:20.048811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62115cc6-5a9f-4f8f-81cc-559fedb0a7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:20.131585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-62115cc6-5a9f-4f8f-81cc-559fedb0a7ba None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:20.132801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-62115cc6-5a9f-4f8f-81cc-559fedb0a7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:20.160556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:20.230212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:20.390324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec73a06-eb82-4ebc-bfce-4f20c0fb7a87 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:20.393069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2ec73a06-eb82-4ebc-bfce-4f20c0fb7a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:20.550732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec73a06-eb82-4ebc-bfce-4f20c0fb7a87 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:20.593186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:20.672955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:20.673489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:20.673763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:20.674028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:20.675654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:20.768945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:20.770575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580f931-4f6d-4278-8495-d302c8e92111 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:20.788789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 711/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:21.674746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822981d4-a8e8-4357-b8ec-ec5cbded45fd None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:21.675336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822981d4-a8e8-4357-b8ec-ec5cbded45fd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:21.675336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822981d4-a8e8-4357-b8ec-ec5cbded45fd None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:21.676582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:21.682760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfaab-7f9b-4330-a34e-249c6a096aeb None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:21.683203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfaab-7f9b-4330-a34e-249c6a096aeb None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:21.683318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfaab-7f9b-4330-a34e-249c6a096aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:21.749454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:21.778367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3f5dfaab-7f9b-4330-a34e-249c6a096aeb None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:21.779587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3f5dfaab-7f9b-4330-a34e-249c6a096aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:21.879096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:22.035318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822981d4-a8e8-4357-b8ec-ec5cbded45fd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:22.036750 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-822981d4-a8e8-4357-b8ec-ec5cbded45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:22.201708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822981d4-a8e8-4357-b8ec-ec5cbded45fd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:22.213930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:22.317427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30959841-82c1-4066-9910-90d6b134937c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:22.317943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30959841-82c1-4066-9910-90d6b134937c service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:22.318248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30959841-82c1-4066-9910-90d6b134937c service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:22.318512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30959841-82c1-4066-9910-90d6b134937c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:22.320096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-30959841-82c1-4066-9910-90d6b134937c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:22.417708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30959841-82c1-4066-9910-90d6b134937c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:22.419359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30959841-82c1-4066-9910-90d6b134937c service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:22.438274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 710/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:23.291557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee10df0-8e5d-4792-ba44-2d8ccc672f82 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:23.292139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee10df0-8e5d-4792-ba44-2d8ccc672f82 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:23.292139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee10df0-8e5d-4792-ba44-2d8ccc672f82 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:23.293499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:23.298810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784fc3-a71b-4d23-9e6e-33e4c598d696 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:23.299237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784fc3-a71b-4d23-9e6e-33e4c598d696 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:23.299353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784fc3-a71b-4d23-9e6e-33e4c598d696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:23.319273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:23.378565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-65784fc3-a71b-4d23-9e6e-33e4c598d696 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:23.379773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-65784fc3-a71b-4d23-9e6e-33e4c598d696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:23.481087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:23.637730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee10df0-8e5d-4792-ba44-2d8ccc672f82 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:23.640120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-aee10df0-8e5d-4792-ba44-2d8ccc672f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:23.767092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:23.804190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee10df0-8e5d-4792-ba44-2d8ccc672f82 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:23.927706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:23.928230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:23.928506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:23.928773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:23.930396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:24.010475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:24.029603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:24.031252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663d10-3bba-4105-89f5-21e7f2328d2b service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:24.050114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 714/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:24.933567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5bfee-e967-4776-90b3-0515c25c4d07 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:24.934542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5bfee-e967-4776-90b3-0515c25c4d07 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:24.934542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5bfee-e967-4776-90b3-0515c25c4d07 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:24.936687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:24.942778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435af7cf-d88c-4f2c-abb2-44e42b45f16e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:24.943514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435af7cf-d88c-4f2c-abb2-44e42b45f16e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:24.943705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435af7cf-d88c-4f2c-abb2-44e42b45f16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:25.026727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-435af7cf-d88c-4f2c-abb2-44e42b45f16e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:25.027947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-435af7cf-d88c-4f2c-abb2-44e42b45f16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:25.256547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:25.408232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5bfee-e967-4776-90b3-0515c25c4d07 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:25.409778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-45f5bfee-e967-4776-90b3-0515c25c4d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:25.502648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:25.572648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5bfee-e967-4776-90b3-0515c25c4d07 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:25.690591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:25.691203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:25.691479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:25.691743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:25.693349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:25.741891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:25.784570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:25.786249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aac0-1e8c-42c1-8fcc-c0f79e4db8a1 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:25.805157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 713/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:26.619327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c066c-52cf-4890-8c35-6b31728a0d27 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:26.619883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c066c-52cf-4890-8c35-6b31728a0d27 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:26.619883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c066c-52cf-4890-8c35-6b31728a0d27 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:26.621925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:26.630787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a8453-88b0-4a38-bcb5-5330e737538a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:26.631233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a8453-88b0-4a38-bcb5-5330e737538a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:26.631827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a8453-88b0-4a38-bcb5-5330e737538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:26.717010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6f6a8453-88b0-4a38-bcb5-5330e737538a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:26.718193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6f6a8453-88b0-4a38-bcb5-5330e737538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:26.815958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:26.966966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c066c-52cf-4890-8c35-6b31728a0d27 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:26.969484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9c6c066c-52cf-4890-8c35-6b31728a0d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:27.031215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:27.138362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c066c-52cf-4890-8c35-6b31728a0d27 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:27.264348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:27.264877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:27.265149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:27.265417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:27.267061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:27.274015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:27.358401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:27.359999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b9e42-7339-4c20-8d2c-b59ff490a980 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:27.379317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 717/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:28.185450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8d1e5-54ce-4e60-8df8-5c7f931e9218 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:28.186546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8d1e5-54ce-4e60-8df8-5c7f931e9218 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:28.186546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8d1e5-54ce-4e60-8df8-5c7f931e9218 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:28.187437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:28.192739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce284ea-d361-4a4c-a585-5775a177a80d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:28.193378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce284ea-d361-4a4c-a585-5775a177a80d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:28.193546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce284ea-d361-4a4c-a585-5775a177a80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:28.275816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5ce284ea-d361-4a4c-a585-5775a177a80d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:28.277047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5ce284ea-d361-4a4c-a585-5775a177a80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:28.345603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:28.376286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:28.535269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8d1e5-54ce-4e60-8df8-5c7f931e9218 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:28.536761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-02a8d1e5-54ce-4e60-8df8-5c7f931e9218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:28.566859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:28.717491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8d1e5-54ce-4e60-8df8-5c7f931e9218 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:28.809779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:28.839979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:28.840597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:28.840869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:28.841134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:28.842777 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:28.934790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:28.936317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9842b7c-fe1f-4cc0-840e-f91a5a783d06 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:28.954546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 716/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:29.768742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27417e-f59b-4df1-aa66-0e864cd17b2d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:29.769199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27417e-f59b-4df1-aa66-0e864cd17b2d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:29.769300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27417e-f59b-4df1-aa66-0e864cd17b2d None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:29.770756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:29.776645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c9b8-c6b5-4086-8f2e-2f01b16620e1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:29.777144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c9b8-c6b5-4086-8f2e-2f01b16620e1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:29.777250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c9b8-c6b5-4086-8f2e-2f01b16620e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:29.855553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4fc6c9b8-c6b5-4086-8f2e-2f01b16620e1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:29.856778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4fc6c9b8-c6b5-4086-8f2e-2f01b16620e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:29.900947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:29.973702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:34:30.127553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27417e-f59b-4df1-aa66-0e864cd17b2d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:30.130043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ff27417e-f59b-4df1-aa66-0e864cd17b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:30.288237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27417e-f59b-4df1-aa66-0e864cd17b2d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:30.347138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:30.423120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:30.423732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:30.424034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:30.424304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:30.425960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:30.527770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:30.529363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8126ee-72d2-4175-bf9f-e297e42801b0 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:30.548306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 720/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:31.348106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939991-7abe-4ca5-9229-9e69bc2ac73e None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:31.349035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939991-7abe-4ca5-9229-9e69bc2ac73e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:31.349035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939991-7abe-4ca5-9229-9e69bc2ac73e None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:31.351511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:34:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:31.359252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7c896-10d5-4d87-a643-969746ff036c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:31.359675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7c896-10d5-4d87-a643-969746ff036c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:31.359817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7c896-10d5-4d87-a643-969746ff036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:31.448206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1fd7c896-10d5-4d87-a643-969746ff036c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:31.449546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1fd7c896-10d5-4d87-a643-969746ff036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:31.451618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:31.560843 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:31.719001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939991-7abe-4ca5-9229-9e69bc2ac73e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:31.720427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b5939991-7abe-4ca5-9229-9e69bc2ac73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:31.874668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939991-7abe-4ca5-9229-9e69bc2ac73e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:31.901194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:31.997497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:31.998028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:31.998313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:31.998608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:32.000202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:32.093061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:32.095531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e09b2-33f0-47fb-a1a4-aa8094b07517 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:32.118572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 719/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:34.026018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1f3c4-df06-43ea-8034-54afda570852 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:34.026960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1f3c4-df06-43ea-8034-54afda570852 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:34.026960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1f3c4-df06-43ea-8034-54afda570852 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:34.028334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:34.033748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca11e8-76d8-44ec-8d2d-7ce495cdce63 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:34.034238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca11e8-76d8-44ec-8d2d-7ce495cdce63 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:34.034355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca11e8-76d8-44ec-8d2d-7ce495cdce63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:34.075196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:34.100923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1eca11e8-76d8-44ec-8d2d-7ce495cdce63 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:34.102130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1eca11e8-76d8-44ec-8d2d-7ce495cdce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:34.200429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:34.239309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1f3c4-df06-43ea-8034-54afda570852 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:34.241123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1aa1f3c4-df06-43ea-8034-54afda570852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:34.349012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1f3c4-df06-43ea-8034-54afda570852 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:34.381740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:34.423748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:34.424365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:34.424642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:34.424909 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:34.426586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:34.536752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:34.538194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e759e-964a-4092-a9fe-6bd738fd2547 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:34.556709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 723/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:36.673680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc003c-1123-4c83-af84-7d0e12168c6c None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:36.674631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc003c-1123-4c83-af84-7d0e12168c6c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:36.674631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc003c-1123-4c83-af84-7d0e12168c6c None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:36.676770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:36.685489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829c814-de21-4ecf-a397-0f3e79404b01 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:36.685946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829c814-de21-4ecf-a397-0f3e79404b01 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:36.686076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829c814-de21-4ecf-a397-0f3e79404b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:36.763522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7829c814-de21-4ecf-a397-0f3e79404b01 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:36.764742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7829c814-de21-4ecf-a397-0f3e79404b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:36.778684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:36.863531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:36.905084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc003c-1123-4c83-af84-7d0e12168c6c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:36.906744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-12fc003c-1123-4c83-af84-7d0e12168c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:37.030640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc003c-1123-4c83-af84-7d0e12168c6c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:37.043436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:37.107000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:37.107616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:37.107896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:37.108167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:37.109778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:37.218649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:37.220094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545b4b9-2a10-46bf-babd-f05e3ab58bcc service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:37.238870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 722/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:39.622318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa71503-e1f6-4a90-95bb-57f6ae3b739c None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:39.622973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa71503-e1f6-4a90-95bb-57f6ae3b739c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:39.622973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa71503-e1f6-4a90-95bb-57f6ae3b739c None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:39.624230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:39.630729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7cc3-cad2-4b76-ab92-5851f44b6e66 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:39.631465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7cc3-cad2-4b76-ab92-5851f44b6e66 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:39.631622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7cc3-cad2-4b76-ab92-5851f44b6e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:39.683385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:39.701443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-340f7cc3-cad2-4b76-ab92-5851f44b6e66 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:39.702705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-340f7cc3-cad2-4b76-ab92-5851f44b6e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:39.812643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:39.852498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa71503-e1f6-4a90-95bb-57f6ae3b739c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:39.854053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1aa71503-e1f6-4a90-95bb-57f6ae3b739c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:39.959386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa71503-e1f6-4a90-95bb-57f6ae3b739c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:40.003708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:40.040019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:40.040626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:40.040899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:40.041166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:40.042815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:40.156009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:40.157460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026487d2-f71c-4a24-8982-6619ab452d45 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:40.176387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 726/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:42.309302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1deb-f60c-4942-baa3-bc3f6db3e52b None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:42.310013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1deb-f60c-4942-baa3-bc3f6db3e52b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:42.310013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1deb-f60c-4942-baa3-bc3f6db3e52b None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:42.311349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:42.318707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e867a04-5418-4aa4-a04b-aef5475f7829 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:42.319473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e867a04-5418-4aa4-a04b-aef5475f7829 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:42.319626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e867a04-5418-4aa4-a04b-aef5475f7829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:42.391143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2e867a04-5418-4aa4-a04b-aef5475f7829 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:42.392340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2e867a04-5418-4aa4-a04b-aef5475f7829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:42.417015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:42.497213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:42.542191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad1deb-f60c-4942-baa3-bc3f6db3e52b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:42.543775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-83ad1deb-f60c-4942-baa3-bc3f6db3e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:42.652519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad1deb-f60c-4942-baa3-bc3f6db3e52b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:42.673991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:42.730807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:42.731453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:42.731755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:42.732025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:42.733633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:42.847646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:42.849108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f68d7-5fbc-44e7-9c64-44cfae543b51 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:42.867878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 725/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:44.892811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bb4b4-b679-46d2-a0d9-3efd4fd95671 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:44.893717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bb4b4-b679-46d2-a0d9-3efd4fd95671 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:44.893717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bb4b4-b679-46d2-a0d9-3efd4fd95671 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:44.895903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:44.902213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11198f-b498-425e-97aa-782464009b13 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:44.902915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11198f-b498-425e-97aa-782464009b13 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:44.903101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11198f-b498-425e-97aa-782464009b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:44.969904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:44.973677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2d11198f-b498-425e-97aa-782464009b13 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:44.977094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2d11198f-b498-425e-97aa-782464009b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:45.086495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:45.125405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bb4b4-b679-46d2-a0d9-3efd4fd95671 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:45.126977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a27bb4b4-b679-46d2-a0d9-3efd4fd95671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:45.234848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bb4b4-b679-46d2-a0d9-3efd4fd95671 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:45.286606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:45.312057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:45.312673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:45.312931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:45.313232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:45.314854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:45.428245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:45.429683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c83cc-27fc-4110-85ba-27c833cb7c86 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:45.448231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 729/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:47.477642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6154-58e6-4a9e-9548-8a986aad9f95 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:47.478225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6154-58e6-4a9e-9548-8a986aad9f95 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:47.478225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6154-58e6-4a9e-9548-8a986aad9f95 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:47.479577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:47.485124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7242c-42f5-4570-8c18-47d04bf261e8 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:47.485592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7242c-42f5-4570-8c18-47d04bf261e8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:47.485697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7242c-42f5-4570-8c18-47d04bf261e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:47.551596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-dab7242c-42f5-4570-8c18-47d04bf261e8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:47.552804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dab7242c-42f5-4570-8c18-47d04bf261e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:47.581210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:47.650568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:47.689027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6154-58e6-4a9e-9548-8a986aad9f95 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:47.691577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2bbf6154-58e6-4a9e-9548-8a986aad9f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:47.803803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6154-58e6-4a9e-9548-8a986aad9f95 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:47.831768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:47.881116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:47.881723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:47.882003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:47.882315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:47.883913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:47.998198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:47.999573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357a387-4fff-472c-b9f6-25fc56d5f08c service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:48.028281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 728/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:50.113364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11ed89-4a01-4198-8eca-2f4d7d20675e None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:50.114278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11ed89-4a01-4198-8eca-2f4d7d20675e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:50.114278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11ed89-4a01-4198-8eca-2f4d7d20675e None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:50.116515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:50.122026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7186f-50bb-4d58-beab-fefb8eb1c208 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:50.122557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7186f-50bb-4d58-beab-fefb8eb1c208 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:50.122662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7186f-50bb-4d58-beab-fefb8eb1c208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:50.194194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2ec7186f-50bb-4d58-beab-fefb8eb1c208 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:50.195452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2ec7186f-50bb-4d58-beab-fefb8eb1c208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:50.204157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:50.299663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:50.340082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11ed89-4a01-4198-8eca-2f4d7d20675e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:50.341797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0f11ed89-4a01-4198-8eca-2f4d7d20675e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:50.462115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11ed89-4a01-4198-8eca-2f4d7d20675e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:50.524363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:50.552189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:50.553057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:50.553381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:50.553739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:50.556197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:50.691081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:50.692792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b917-9d5d-4eee-b0b9-aace83d51aaa service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:50.714408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 732/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:52.752821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb780d7-0bef-4ec3-903c-6edf0a454015 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:52.753391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb780d7-0bef-4ec3-903c-6edf0a454015 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:52.753391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb780d7-0bef-4ec3-903c-6edf0a454015 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:52.755360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:52.764274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4cd0f-f70a-4096-b4c1-ed78747fc0ba None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:52.764701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4cd0f-f70a-4096-b4c1-ed78747fc0ba None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:52.764894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4cd0f-f70a-4096-b4c1-ed78747fc0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:52.837180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3bd4cd0f-f70a-4096-b4c1-ed78747fc0ba None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:52.838430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3bd4cd0f-f70a-4096-b4c1-ed78747fc0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:52.878297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:52.941216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:52.979458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb780d7-0bef-4ec3-903c-6edf0a454015 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:52.980997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7fb780d7-0bef-4ec3-903c-6edf0a454015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:53.085803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb780d7-0bef-4ec3-903c-6edf0a454015 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:53.125286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:53.163011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:53.163617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:53.163893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:53.164163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:53.165756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:53.280480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:53.281921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fca7b-c408-40e0-99e9-fc6054ade10b service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:53.300976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 731/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:55.303480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac12e9-6a90-446f-ac61-1c454e3292b2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:55.304058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac12e9-6a90-446f-ac61-1c454e3292b2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:55.304058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac12e9-6a90-446f-ac61-1c454e3292b2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:55.306115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:55.313168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57acc0-f599-4070-a966-bdcd838edd76 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:55.313913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57acc0-f599-4070-a966-bdcd838edd76 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:55.314138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57acc0-f599-4070-a966-bdcd838edd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:55.390222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-9e57acc0-f599-4070-a966-bdcd838edd76 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:55.391451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9e57acc0-f599-4070-a966-bdcd838edd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:55.410780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:55.497364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:55.534252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac12e9-6a90-446f-ac61-1c454e3292b2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:55.535805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5bac12e9-6a90-446f-ac61-1c454e3292b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:55.650776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac12e9-6a90-446f-ac61-1c454e3292b2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:55.728993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:55.737107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:55.737722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:55.737996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:55.738316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:55.739937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:55.854703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:55.856162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2244c6-c62d-4838-bbdb-62047bdbc8dd service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:55.874035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 735/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:34:57.861279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed36e0e-f95e-4948-a92b-c36a885a9409 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:57.861851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed36e0e-f95e-4948-a92b-c36a885a9409 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:57.861851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed36e0e-f95e-4948-a92b-c36a885a9409 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:57.863250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:34:57.869813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105829-62af-4cca-bf8a-d87d3773c20b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:57.870621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105829-62af-4cca-bf8a-d87d3773c20b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:57.870765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105829-62af-4cca-bf8a-d87d3773c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:57.939673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c4105829-62af-4cca-bf8a-d87d3773c20b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:34:57.940889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c4105829-62af-4cca-bf8a-d87d3773c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:57.995620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:58.046227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:34:58.083559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed36e0e-f95e-4948-a92b-c36a885a9409 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:34:58.085988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8ed36e0e-f95e-4948-a92b-c36a885a9409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:58.207722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed36e0e-f95e-4948-a92b-c36a885a9409 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:34:58.256788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:34:58.284120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:34:58.284718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:34:58.284995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:34:58.285260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:34:58.286878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:34:58.398517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:34:58.399949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508f00b-6f74-4a9b-9677-d35881f472f4 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:34:58.422694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 734/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:35:08.044408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=22 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:08.419874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=66 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:17.764173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0207b7-443f-497c-9671-b70c57fa86af None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:17.764772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0207b7-443f-497c-9671-b70c57fa86af None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:17.764772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0207b7-443f-497c-9671-b70c57fa86af None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:17.765979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 737/1471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:17.772546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fae2-4ee7-4f86-940d-4ae509b0660d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:17.773286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fae2-4ee7-4f86-940d-4ae509b0660d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:17.773437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fae2-4ee7-4f86-940d-4ae509b0660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:17.850349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b867fae2-4ee7-4f86-940d-4ae509b0660d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:17.851555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b867fae2-4ee7-4f86-940d-4ae509b0660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:17.968760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 735/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:18.060805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a797b09-8892-4c1f-9bb5-0ef320317600 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:18.061084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a797b09-8892-4c1f-9bb5-0ef320317600 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:18.061353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a797b09-8892-4c1f-9bb5-0ef320317600 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:18.062743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 738/1473] 10.222.0.20 () {54 vars in 967 bytes} [Tue Aug 9 02:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:18.068498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f6d1e-e088-4fb6-acc5-f05cd69f3c23 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:18.068869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f6d1e-e088-4fb6-acc5-f05cd69f3c23 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:18.069238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f6d1e-e088-4fb6-acc5-f05cd69f3c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:18.154605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-265f6d1e-e088-4fb6-acc5-f05cd69f3c23 None None] MFA Rules not processed for user `7a0b055ed0d14d3e9428d1291f0f1209`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:18.155852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-265f6d1e-e088-4fb6-acc5-f05cd69f3c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:18.177364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:18.240271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 736/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Tue Aug 9 02:35:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:18.248292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a797b09-8892-4c1f-9bb5-0ef320317600 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:18.249800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0a797b09-8892-4c1f-9bb5-0ef320317600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:18.361931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a797b09-8892-4c1f-9bb5-0ef320317600 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:18.443553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:18.444598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:18.445254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:18.445532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:18.447175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:18.502285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:18.558677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:18.560157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84b2b9-ea46-41d5-bb50-aa53284cd197 service glance] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:18.578863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 739/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:35:20.656696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb16476-4ce9-437f-a314-ad4b681291c9 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:20.657554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb16476-4ce9-437f-a314-ad4b681291c9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:20.657554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb16476-4ce9-437f-a314-ad4b681291c9 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:20.659857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 737/1476] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:20.666228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f0a6c-951d-428f-ae3e-bc8d36d40e19 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:20.666662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f0a6c-951d-428f-ae3e-bc8d36d40e19 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:20.666766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f0a6c-951d-428f-ae3e-bc8d36d40e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:20.743729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-406f0a6c-951d-428f-ae3e-bc8d36d40e19 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:20.744949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-406f0a6c-951d-428f-ae3e-bc8d36d40e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:20.858188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 740/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:20.881005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb16476-4ce9-437f-a314-ad4b681291c9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:20.882596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0cb16476-4ce9-437f-a314-ad4b681291c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:21.024220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb16476-4ce9-437f-a314-ad4b681291c9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:21.073194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:21.121161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:21.121855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:21.122138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:21.122444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:21.123103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:21.124846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:35:21.124846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:35:21.124846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:35:21.126005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0103e6-1fba-47bb-bc36-f06d7def8dba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:21.145526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 738/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 9 02:35:20 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:35:23.151126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cca998-749d-4d97-81dc-a10500bf921d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:23.151743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cca998-749d-4d97-81dc-a10500bf921d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:23.151743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cca998-749d-4d97-81dc-a10500bf921d None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:23.153019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 741/1479] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:23.159329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf7e93-a2aa-438f-be69-b19f1bdf9f8f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:23.159784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf7e93-a2aa-438f-be69-b19f1bdf9f8f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:23.159897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf7e93-a2aa-438f-be69-b19f1bdf9f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:23.235633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f8cf7e93-a2aa-438f-be69-b19f1bdf9f8f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:23.236850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f8cf7e93-a2aa-438f-be69-b19f1bdf9f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:23.337999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 739/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:23.383744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cca998-749d-4d97-81dc-a10500bf921d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:23.385880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-84cca998-749d-4d97-81dc-a10500bf921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:23.401807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:23.548235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cca998-749d-4d97-81dc-a10500bf921d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:23.642400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:23.686498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:23.687036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:23.687307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:23.687576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:23.689156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:23.781167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:23.782931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac185-dc65-4e78-8ba5-9636f655342c service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:23.802403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 742/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:35:25.994968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cc60b-2e22-4244-9123-3018f3c1980b None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:25.995572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cc60b-2e22-4244-9123-3018f3c1980b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:25.995572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cc60b-2e22-4244-9123-3018f3c1980b None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:25.996893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 740/1482] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:26.003419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c184-5552-40b6-bd17-9f076f13e58c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:26.004150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c184-5552-40b6-bd17-9f076f13e58c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:26.004337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c184-5552-40b6-bd17-9f076f13e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:26.074877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0740c184-5552-40b6-bd17-9f076f13e58c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:26.076079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0740c184-5552-40b6-bd17-9f076f13e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:26.136297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:26.189419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 743/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:26.229167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cc60b-2e22-4244-9123-3018f3c1980b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:26.230797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4b4cc60b-2e22-4244-9123-3018f3c1980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:26.257281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:26.342207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cc60b-2e22-4244-9123-3018f3c1980b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:26.422904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:26.423524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:26.423798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:26.424045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:26.425663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:26.509744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:26.547891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:26.549343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1aa0d0-03c6-4d7d-ab7b-06c95760265c service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:26.568484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 741/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:35:28.619220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35790b2-e61b-4b72-adaa-57e8d77a3802 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:28.620040 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35790b2-e61b-4b72-adaa-57e8d77a3802 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:28.620154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35790b2-e61b-4b72-adaa-57e8d77a3802 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:28.622223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 744/1485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:28.628287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94318c-7067-420c-9d2c-6bcd14e34907 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:28.628721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94318c-7067-420c-9d2c-6bcd14e34907 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:28.628820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94318c-7067-420c-9d2c-6bcd14e34907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:28.702444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6e94318c-7067-420c-9d2c-6bcd14e34907 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:28.703597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6e94318c-7067-420c-9d2c-6bcd14e34907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:28.823075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 742/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:28.865271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35790b2-e61b-4b72-adaa-57e8d77a3802 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:28.867857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f35790b2-e61b-4b72-adaa-57e8d77a3802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:28.984918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35790b2-e61b-4b72-adaa-57e8d77a3802 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:29.074446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:29.077174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:29.077817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:29.078104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:29.078410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:29.080773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:29.202359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:29.203805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c09732-8d94-4c8e-90db-c7f3dd9b17aa service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:29.222906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 745/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:35:31.340222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba42dc-d4ca-49b8-9fa2-f9fe25c41bc2 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:31.341185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba42dc-d4ca-49b8-9fa2-f9fe25c41bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:31.341185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba42dc-d4ca-49b8-9fa2-f9fe25c41bc2 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:31.343368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 743/1488] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:31.350009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef738d-bb3a-453e-8648-601c9608db01 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:31.350804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef738d-bb3a-453e-8648-601c9608db01 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:31.350982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef738d-bb3a-453e-8648-601c9608db01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:31.424744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7cef738d-bb3a-453e-8648-601c9608db01 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:31.425973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7cef738d-bb3a-453e-8648-601c9608db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:31.480921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:31.526885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 746/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:35:31.572664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba42dc-d4ca-49b8-9fa2-f9fe25c41bc2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:31.574215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d0ba42dc-d4ca-49b8-9fa2-f9fe25c41bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:31.615678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:31.687791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba42dc-d4ca-49b8-9fa2-f9fe25c41bc2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:31.774388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:31.775011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:31.775292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:31.775562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:31.777189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:31.866286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:31.901276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:31.902860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6fa48-3cf5-4473-b895-a4804e571a6c service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:31.921261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 744/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:35:34.133136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486431-0533-4a33-a63f-a056274b04bc None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:34.133729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486431-0533-4a33-a63f-a056274b04bc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:34.133729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486431-0533-4a33-a63f-a056274b04bc None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:34.135808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 747/1491] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 9 02:35:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:35:34.142892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d75655-7c10-4946-b845-83b12285cead None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:34.143375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d75655-7c10-4946-b845-83b12285cead None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:34.143480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d75655-7c10-4946-b845-83b12285cead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:34.215957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b8d75655-7c10-4946-b845-83b12285cead None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:35:34.217179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b8d75655-7c10-4946-b845-83b12285cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:34.320956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 745/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 9 02:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:35:34.363078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486431-0533-4a33-a63f-a056274b04bc None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:35:34.364663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-57486431-0533-4a33-a63f-a056274b04bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:34.472291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486431-0533-4a33-a63f-a056274b04bc None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:35:34.549963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:35:34.550607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:35:34.550886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:35:34.551157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:35:34.552800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:35:34.562727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:34.671543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:35:34.672908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581912f-5758-4776-adea-81d1cb6eb0f3 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:35:34.691101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 748/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:35:44.317258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=82 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:35:44.688848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=50 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:00.151093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:00.152031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:00.152031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:00.247820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:37:00.249036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:00.356096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 746/1494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:37:00.372384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:00.372920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:00.372920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:00.453351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:37:00.454601 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:00.553585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 749/1495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:37:00.558970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:00.560285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:00.572600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45cf6-6a61-4eff-b0a3-69129198d650 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:00.638479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:00.671286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:00.671833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:00.672112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:00.672365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:00.673032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:37:00.673331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:00.674392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:00.674392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:37:00.674392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:37:00.675330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:00.687444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 747/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:37:00.699921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:00.709432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d755d7-bb13-4047-b62c-0acf7a9b1422 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:00.772749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:00.806625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:00.807249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:00.807527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:00.807802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:00.808457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:00.810230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:00.810230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:37:00.810230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:37:00.811429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:00.875522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 750/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:37:00 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:37:00.882135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:00.883731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:00.916579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dfc37-cafe-49af-bf1b-198379aa6831 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:01.018245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:01.018786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.019068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.019339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.019979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:01.020990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:01.020990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:37:01.020990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:37:01.022237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:01.068539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 748/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:37:00 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:37:01.072994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:01.074585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:01.136833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4536571-7642-45dd-a4be-3672483f441b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:01.223045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:01.236477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:01.237094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.237389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.237641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.238317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:01.239340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:01.239340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:37:01.239340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:37:01.240256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:01.253228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 751/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:37:01.257599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:01.259173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:01.271017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11870f5-4567-41e3-bdf0-e57711f8cabc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:01.296836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:01.368301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:01.368901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.369177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.369448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.370091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:01.371115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:01.371115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:37:01.371115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:37:01.372018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b4457-c098-4fda-b6bb-99e266d25043 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:01.385830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 749/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:37:01.390408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:01.392048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:01.428685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c65d-7b6f-4fab-88bc-056e52f57744 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:01.527450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:01.528129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.528398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.528662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] PATH_INFO: `/v3/projects/b5b97b43554d40a286cbbb41853ed540/users/aad6583efe49404ca6480008afda13f5/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.550858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b97b43554d40a286cbbb41853ed540, user_id=aad6583efe49404ca6480008afda13f5, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:01.551887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:01.551887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:37:01.551887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:37:01.553835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987d169-1939-4a2b-a6c9-4b35501e319f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:01.580138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 752/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:37:01 2022] PUT /identity/v3/projects/b5b97b43554d40a286cbbb41853ed540/users/aad6583efe49404ca6480008afda13f5/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:37:01.592688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9972b6-55d6-4f6c-87d4-0d2bd9ce2758 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.592976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9972b6-55d6-4f6c-87d4-0d2bd9ce2758 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.593270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9972b6-55d6-4f6c-87d4-0d2bd9ce2758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.656585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-fb9972b6-55d6-4f6c-87d4-0d2bd9ce2758 None None] MFA Rules not processed for user `aad6583efe49404ca6480008afda13f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:37:01.658436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fb9972b6-55d6-4f6c-87d4-0d2bd9ce2758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:01.767780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 750/1502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:37:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:37:01.781816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae349f1-795c-4d2f-a0e5-9d36736cadf4 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.782099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae349f1-795c-4d2f-a0e5-9d36736cadf4 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.782392 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae349f1-795c-4d2f-a0e5-9d36736cadf4 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.783724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 753/1503] 10.222.0.20 () {54 vars in 964 bytes} [Tue Aug 9 02:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:37:01.788673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d7c40-2131-436d-825c-c6108efb9581 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:01.788963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d7c40-2131-436d-825c-c6108efb9581 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:01.789243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d7c40-2131-436d-825c-c6108efb9581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:01.820761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:01.868769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-275d7c40-2131-436d-825c-c6108efb9581 None None] MFA Rules not processed for user `d57751a2455c48d28fcf9e97b62cfc38`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:37:01.869990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-275d7c40-2131-436d-825c-c6108efb9581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:01.964331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 751/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Aug 9 02:37:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:37:01.972329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae349f1-795c-4d2f-a0e5-9d36736cadf4 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:01.973934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9ae349f1-795c-4d2f-a0e5-9d36736cadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:02.027705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:02.083057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae349f1-795c-4d2f-a0e5-9d36736cadf4 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:02.166338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:02.166949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:02.167225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:02.167486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:02.169069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:02.267750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:02.269191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:02.272835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:02.291128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 754/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:37:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:37:02.338030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aef668-ed9c-4ae3-990f-225838fef3c5 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:02.338530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aef668-ed9c-4ae3-990f-225838fef3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:02.338641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aef668-ed9c-4ae3-990f-225838fef3c5 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:02.339941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 752/1506] 10.222.0.20 () {52 vars in 830 bytes} [Tue Aug 9 02:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:37:02.343672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:02.344894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:02.450293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a9bcb-340c-4548-a9d6-eea1a64d8693 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:02.551524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:02.552080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:02.552358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:02.552625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] PATH_INFO: `/v3/users/aad6583efe49404ca6480008afda13f5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:02.571171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad6583efe49404ca6480008afda13f5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:02.572215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:02.572215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:37:02.572215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:37:02.573437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:02.583823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:02.645971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad6583efe49404ca6480008afda13f5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:02.646425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad6583efe49404ca6480008afda13f5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:02.658645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad6583efe49404ca6480008afda13f5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:02.672602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad6583efe49404ca6480008afda13f5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:02.695119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad6583efe49404ca6480008afda13f5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:02.700784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 755/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:37:02 2022] DELETE /identity/v3/users/aad6583efe49404ca6480008afda13f5 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:37:02.705318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:02.705606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:02.705878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:02.763484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:37:02.764692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:02.846557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:02.859545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 753/1508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:37:02.870138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:02.871430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:02.976262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b96272-0df9-46fa-9441-aa89835038bd admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:03.038386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:03.067210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:03.067839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:03.068117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:03.068386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:03.070962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:03.186249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:03.187702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e54f-47bb-42c2-9716-1ce24c157786 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:03.206778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 756/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:37:03.560729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:37:03.562000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:37:03.573537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf7c13-d729-4955-81b8-05b09b1c638c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:37:03.670618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:37:03.671293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:37:03.671569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:37:03.671863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] PATH_INFO: `/v3/projects/b5b97b43554d40a286cbbb41853ed540` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:37:03.682413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b97b43554d40a286cbbb41853ed540)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:37:03.683468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:37:03.683468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:37:03.683468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:37:03.684671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:37:03.754320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b97b43554d40a286cbbb41853ed540', 'request_id': 'req-cc263939-73d2-4809-9127-b6fce90b0d45'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:03.764389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b97b43554d40a286cbbb41853ed540', 'request_id': 'req-cc263939-73d2-4809-9127-b6fce90b0d45'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:03.777697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b97b43554d40a286cbbb41853ed540', 'request_id': 'req-cc263939-73d2-4809-9127-b6fce90b0d45'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:03.800321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] The token cache is being invalidate because project b5b97b43554d40a286cbbb41853ed540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:37:03.800621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:03.801128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc263939-73d2-4809-9127-b6fce90b0d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:37:03.802467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 754/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:37:03 2022] DELETE /identity/v3/projects/b5b97b43554d40a286cbbb41853ed540 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:37:13.204668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=62 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:37:13.798541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:38:28.725615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13195-9596-43af-a94d-1509c0b915ec None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:28.726632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13195-9596-43af-a94d-1509c0b915ec None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:28.726632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13195-9596-43af-a94d-1509c0b915ec None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:28.727835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:28.733905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094c77a-6754-4e83-9c11-45a93420c1db None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:28.734431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094c77a-6754-4e83-9c11-45a93420c1db None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:28.734541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094c77a-6754-4e83-9c11-45a93420c1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:28.825322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1094c77a-6754-4e83-9c11-45a93420c1db None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:28.826565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1094c77a-6754-4e83-9c11-45a93420c1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:28.935256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:29.090946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13195-9596-43af-a94d-1509c0b915ec None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:29.095027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f4d13195-9596-43af-a94d-1509c0b915ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:29.220485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13195-9596-43af-a94d-1509c0b915ec None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:29.226449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6727cb-70c5-4122-8af2-3e61449900e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:29.227052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6727cb-70c5-4122-8af2-3e61449900e4 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:29.227332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6727cb-70c5-4122-8af2-3e61449900e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:29.227605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6727cb-70c5-4122-8af2-3e61449900e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:29.236302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6727cb-70c5-4122-8af2-3e61449900e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:29.237891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6727cb-70c5-4122-8af2-3e61449900e4 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:29.248065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 758/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:30.079363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7a42-c47c-4c8c-a67a-9e832d84e07a None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:30.079995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7a42-c47c-4c8c-a67a-9e832d84e07a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:30.079995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e7a42-c47c-4c8c-a67a-9e832d84e07a None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:30.081299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:30.086377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa78fa-12bf-4ea1-82b1-02128b0ae68e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:30.086834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa78fa-12bf-4ea1-82b1-02128b0ae68e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:30.086945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa78fa-12bf-4ea1-82b1-02128b0ae68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:30.111874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3ffa78fa-12bf-4ea1-82b1-02128b0ae68e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:30.113096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3ffa78fa-12bf-4ea1-82b1-02128b0ae68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:30.122135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:30.278060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7a42-c47c-4c8c-a67a-9e832d84e07a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:30.291035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2cb0d-bf36-45b9-b448-049b9ede7a4d None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:30.291326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2cb0d-bf36-45b9-b448-049b9ede7a4d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:30.291618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2cb0d-bf36-45b9-b448-049b9ede7a4d None None] PATH_INFO: `/v3` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:30.292898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 760/1516] 10.222.0.13 () {54 vars in 925 bytes} [Tue Aug 9 02:38:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:38:30.294027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e7a42-c47c-4c8c-a67a-9e832d84e07a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:30.299244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6282c-e070-4e00-b502-b67b7dde7ae6 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:30.299515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6282c-e070-4e00-b502-b67b7dde7ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:30.299780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6282c-e070-4e00-b502-b67b7dde7ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:30.301664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec5422-c83d-4f78-80cb-0b39c7d20dfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:30.302333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5422-c83d-4f78-80cb-0b39c7d20dfe service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:30.302605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5422-c83d-4f78-80cb-0b39c7d20dfe service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:30.302877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5422-c83d-4f78-80cb-0b39c7d20dfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:30.311460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5422-c83d-4f78-80cb-0b39c7d20dfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:30.313041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5422-c83d-4f78-80cb-0b39c7d20dfe service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:30.322546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 757/1517] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:30.361648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-28b6282c-e070-4e00-b502-b67b7dde7ae6 None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:30.362931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-28b6282c-e070-4e00-b502-b67b7dde7ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:30.426279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:38:30.445143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 761/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Aug 9 02:38:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:39.392127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92e84-727e-4617-ade8-cf8bd9754dbd None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:39.392723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92e84-727e-4617-ade8-cf8bd9754dbd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:39.392723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92e84-727e-4617-ade8-cf8bd9754dbd None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:39.393974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 758/1519] 10.222.0.20 () {54 vars in 961 bytes} [Tue Aug 9 02:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:39.400482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a307f0-5ddb-4363-9f70-bce1e6c05928 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:39.401300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a307f0-5ddb-4363-9f70-bce1e6c05928 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:39.401468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a307f0-5ddb-4363-9f70-bce1e6c05928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:39.424636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d0a307f0-5ddb-4363-9f70-bce1e6c05928 None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:39.425848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d0a307f0-5ddb-4363-9f70-bce1e6c05928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:39.435346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 762/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Aug 9 02:38:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:39.443359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92e84-727e-4617-ade8-cf8bd9754dbd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:39.458531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92e84-727e-4617-ade8-cf8bd9754dbd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:39.467347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387ff66-2a95-45f6-ba9c-f88def586a84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:39.467946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387ff66-2a95-45f6-ba9c-f88def586a84 service placement] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:39.468225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387ff66-2a95-45f6-ba9c-f88def586a84 service placement] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:39.468501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387ff66-2a95-45f6-ba9c-f88def586a84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:39.477437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387ff66-2a95-45f6-ba9c-f88def586a84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:39.479057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387ff66-2a95-45f6-ba9c-f88def586a84 service placement] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:39.488816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 759/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:41.560733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc11d2-6ebe-4a71-88e2-6962e3b12d2a None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:41.561335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc11d2-6ebe-4a71-88e2-6962e3b12d2a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:41.561335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc11d2-6ebe-4a71-88e2-6962e3b12d2a None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:41.562732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:41.568804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917824cb-72f8-4156-a21b-3b45b3e25960 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:41.569559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917824cb-72f8-4156-a21b-3b45b3e25960 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:41.569749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917824cb-72f8-4156-a21b-3b45b3e25960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:41.593604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-917824cb-72f8-4156-a21b-3b45b3e25960 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:41.594844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-917824cb-72f8-4156-a21b-3b45b3e25960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:41.605026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:41.763679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc11d2-6ebe-4a71-88e2-6962e3b12d2a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:41.779439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc11d2-6ebe-4a71-88e2-6962e3b12d2a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:41.785877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da436b5c-0223-495e-985a-c5ee8e581faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:41.786518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da436b5c-0223-495e-985a-c5ee8e581faa service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:41.786798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da436b5c-0223-495e-985a-c5ee8e581faa service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:41.787072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da436b5c-0223-495e-985a-c5ee8e581faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:41.795838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da436b5c-0223-495e-985a-c5ee8e581faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:41.797488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da436b5c-0223-495e-985a-c5ee8e581faa service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:41.806901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 764/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:42.636949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b6e2c-6173-40d8-96b4-a6c8f70cef42 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:42.637546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b6e2c-6173-40d8-96b4-a6c8f70cef42 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:42.637546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b6e2c-6173-40d8-96b4-a6c8f70cef42 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:42.639503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 02:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:42.645424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd0fe-6b1f-44b3-a6c1-09649db2d629 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:42.645863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd0fe-6b1f-44b3-a6c1-09649db2d629 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:42.646001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd0fe-6b1f-44b3-a6c1-09649db2d629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:42.675112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6eccd0fe-6b1f-44b3-a6c1-09649db2d629 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:42.676253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6eccd0fe-6b1f-44b3-a6c1-09649db2d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:42.685144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 02:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:42.832383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b6e2c-6173-40d8-96b4-a6c8f70cef42 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:42.848403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b6e2c-6173-40d8-96b4-a6c8f70cef42 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:42.857521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1dace-1fc7-48a1-b51e-88b756f217b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:42.858480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1dace-1fc7-48a1-b51e-88b756f217b5 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:42.858868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1dace-1fc7-48a1-b51e-88b756f217b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:42.859266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1dace-1fc7-48a1-b51e-88b756f217b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:42.868318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1dace-1fc7-48a1-b51e-88b756f217b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:42.869955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1dace-1fc7-48a1-b51e-88b756f217b5 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:42.881435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 762/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:46.688233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5009-9c78-40c0-8325-683d699e61a8 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:46.689096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5009-9c78-40c0-8325-683d699e61a8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:46.689096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5009-9c78-40c0-8325-683d699e61a8 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:46.691096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:38:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:46.697359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4e851-50f4-4516-9d10-13f1f21437c1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:46.698107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4e851-50f4-4516-9d10-13f1f21437c1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:46.698227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4e851-50f4-4516-9d10-13f1f21437c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:46.727385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-60e4e851-50f4-4516-9d10-13f1f21437c1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:46.728596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-60e4e851-50f4-4516-9d10-13f1f21437c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:46.738185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:46.815967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5009-9c78-40c0-8325-683d699e61a8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:46.819328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-83de5009-9c78-40c0-8325-683d699e61a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:46.904982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5009-9c78-40c0-8325-683d699e61a8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:46.909536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f41c29-3b26-437d-82ea-f65dc3d2b164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:46.910166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f41c29-3b26-437d-82ea-f65dc3d2b164 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:46.910448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f41c29-3b26-437d-82ea-f65dc3d2b164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:46.910712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f41c29-3b26-437d-82ea-f65dc3d2b164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:46.920164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f41c29-3b26-437d-82ea-f65dc3d2b164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:46.921590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f41c29-3b26-437d-82ea-f65dc3d2b164 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:46.931120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 767/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:38:48.928482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a925e8-9b5d-4ba5-93e8-7fb827914244 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:48.929329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a925e8-9b5d-4ba5-93e8-7fb827914244 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:48.929487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a925e8-9b5d-4ba5-93e8-7fb827914244 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:48.931663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:38:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:48.937913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a54ff0-f9ac-46b9-b95b-75c29afce52b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:48.938431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a54ff0-f9ac-46b9-b95b-75c29afce52b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:48.938582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a54ff0-f9ac-46b9-b95b-75c29afce52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:48.961105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-44a54ff0-f9ac-46b9-b95b-75c29afce52b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:48.962305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-44a54ff0-f9ac-46b9-b95b-75c29afce52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:48.971515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:49.011608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a925e8-9b5d-4ba5-93e8-7fb827914244 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:49.026824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a925e8-9b5d-4ba5-93e8-7fb827914244 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:49.031577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1ff89-df49-470f-8841-1fef33901687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:49.032191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1ff89-df49-470f-8841-1fef33901687 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:49.032475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1ff89-df49-470f-8841-1fef33901687 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:49.032749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1ff89-df49-470f-8841-1fef33901687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:49.043187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1ff89-df49-470f-8841-1fef33901687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:49.044642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1ff89-df49-470f-8841-1fef33901687 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:49.054473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 765/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:52.609476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab041b8c-61c8-4a05-b21f-bf22f97c0a22 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:52.610431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab041b8c-61c8-4a05-b21f-bf22f97c0a22 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:52.610431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab041b8c-61c8-4a05-b21f-bf22f97c0a22 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:52.612560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:38:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:52.618897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92979748-6b82-44cd-84f8-5dd90b079451 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:52.619695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92979748-6b82-44cd-84f8-5dd90b079451 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:52.619859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92979748-6b82-44cd-84f8-5dd90b079451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:52.630557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:38:52.643980 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-92979748-6b82-44cd-84f8-5dd90b079451 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:52.645184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-92979748-6b82-44cd-84f8-5dd90b079451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:52.653915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:52.961105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab041b8c-61c8-4a05-b21f-bf22f97c0a22 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:52.963407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ab041b8c-61c8-4a05-b21f-bf22f97c0a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:53.049261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab041b8c-61c8-4a05-b21f-bf22f97c0a22 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:53.053415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f97b6b-ce44-41a7-82a6-e15c63eae70c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:53.054022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97b6b-ce44-41a7-82a6-e15c63eae70c service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:53.054312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97b6b-ce44-41a7-82a6-e15c63eae70c service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:53.054626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97b6b-ce44-41a7-82a6-e15c63eae70c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:53.064759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f97b6b-ce44-41a7-82a6-e15c63eae70c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:53.066187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f97b6b-ce44-41a7-82a6-e15c63eae70c service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:53.075780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 770/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:38:55.122586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674d428-2bf8-4068-9930-1d980d3be53f None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:55.123479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674d428-2bf8-4068-9930-1d980d3be53f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:55.123479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674d428-2bf8-4068-9930-1d980d3be53f None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:55.125632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 02:38:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:38:55.134953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237a60b-fbd4-418a-8f3c-fe3b6282eb72 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:55.135421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237a60b-fbd4-418a-8f3c-fe3b6282eb72 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:55.135421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237a60b-fbd4-418a-8f3c-fe3b6282eb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:55.158690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5237a60b-fbd4-418a-8f3c-fe3b6282eb72 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:38:55.159918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5237a60b-fbd4-418a-8f3c-fe3b6282eb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:55.169333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 02:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:38:55.471138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674d428-2bf8-4068-9930-1d980d3be53f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:38:55.473088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d674d428-2bf8-4068-9930-1d980d3be53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:38:55.488378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674d428-2bf8-4068-9930-1d980d3be53f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:38:55.492282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324f82e-5207-4a75-923c-fa840909b52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:38:55.492880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324f82e-5207-4a75-923c-fa840909b52d service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:38:55.493151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324f82e-5207-4a75-923c-fa840909b52d service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:38:55.493415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324f82e-5207-4a75-923c-fa840909b52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:38:55.502798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324f82e-5207-4a75-923c-fa840909b52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:38:55.504331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324f82e-5207-4a75-923c-fa840909b52d service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:38:55.514067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 768/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:39:05.153737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:39:05.509839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=8 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:17.100870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.101560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.101560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.125902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.127220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.136623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 772/1540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.146265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06326f4-e83c-49f6-ab71-9f2d84e783ec None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.149202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06326f4-e83c-49f6-ab71-9f2d84e783ec None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.149202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06326f4-e83c-49f6-ab71-9f2d84e783ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.181294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b06326f4-e83c-49f6-ab71-9f2d84e783ec None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.182722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b06326f4-e83c-49f6-ab71-9f2d84e783ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.191657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 769/1541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.196690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.201117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2386ed-aaab-4973-89da-18ff11222d85 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.201460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2386ed-aaab-4973-89da-18ff11222d85 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.201676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2386ed-aaab-4973-89da-18ff11222d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.213663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14556-cf86-4f64-98a9-43a905be2f25 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.223238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.223939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.224298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.224550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.225219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.225525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.226599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:40:17.226599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:40:17.226599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:40:17.227501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.230897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5c2386ed-aaab-4973-89da-18ff11222d85 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.232324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5c2386ed-aaab-4973-89da-18ff11222d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.245613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 770/1542] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:40:17.256356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.256711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.256899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.257187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 773/1543] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:17.266224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.285045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bc86-a1a2-491b-8f62-55d7ca5e8e71 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.295628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.297203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.304037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803e81-91a2-4a2e-b43b-f5f3986ecaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.304701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803e81-91a2-4a2e-b43b-f5f3986ecaef admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.304987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803e81-91a2-4a2e-b43b-f5f3986ecaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.305267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803e81-91a2-4a2e-b43b-f5f3986ecaef admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.305943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803e81-91a2-4a2e-b43b-f5f3986ecaef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.307955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803e81-91a2-4a2e-b43b-f5f3986ecaef admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.310432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 771/1544] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.315650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.332378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4962ed-90ac-4648-bcba-9be7517e8a34 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.340879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.341519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.341797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.342068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.343144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.343450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.345097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.370219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 772/1545] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:40:17.382211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.397757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689854e-16fb-40d2-b47e-7b45f53d7ab7 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.400858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 774/1546] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:40:17.402364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1fd0b-3257-43f9-918d-970b9aa306c9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.403069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1fd0b-3257-43f9-918d-970b9aa306c9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.403533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1fd0b-3257-43f9-918d-970b9aa306c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.409310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.410044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.412689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.412689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.412689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.412689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:40:17.412689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:40:17.412689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:40:17.413869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.443653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5fd1fd0b-3257-43f9-918d-970b9aa306c9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.445030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5fd1fd0b-3257-43f9-918d-970b9aa306c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.486650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 773/1547] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:17.487483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.553839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 775/1548] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.555169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.555453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.555725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.590958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.592180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.597801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc18b0-14e6-4dc0-a728-bf3a7fb80084 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.600377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 776/1549] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.601136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.603022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d797fc4-8078-4313-95c1-ac8952c8f4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.603654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d797fc4-8078-4313-95c1-ac8952c8f4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.603926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d797fc4-8078-4313-95c1-ac8952c8f4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.604199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d797fc4-8078-4313-95c1-ac8952c8f4c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.604831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d797fc4-8078-4313-95c1-ac8952c8f4c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.607591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d797fc4-8078-4313-95c1-ac8952c8f4c2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.610872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d563a-c863-42fe-9b92-cb02d662a11c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.615512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.616109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.616386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.617071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.617278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.618326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:40:17.618326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:40:17.618326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:40:17.619517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ba92d-8951-40f6-a44b-2b6e5d3fff8d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.631367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 774/1550] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:17.636490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c18b-f672-4bd5-985f-6f25cd3b54cd None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.636490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c18b-f672-4bd5-985f-6f25cd3b54cd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.636490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c18b-f672-4bd5-985f-6f25cd3b54cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.646736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 777/1551] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:17.648125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e0781-e66d-429c-a23e-d4d0c7bbcc73 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.648421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e0781-e66d-429c-a23e-d4d0c7bbcc73 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.648701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e0781-e66d-429c-a23e-d4d0c7bbcc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.655246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-32b0c18b-f672-4bd5-985f-6f25cd3b54cd None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.656453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-32b0c18b-f672-4bd5-985f-6f25cd3b54cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.665142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 775/1552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.666375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44736-a861-4592-86f2-2d59882a846b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.666659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44736-a861-4592-86f2-2d59882a846b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.666931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44736-a861-4592-86f2-2d59882a846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.669390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-048e0781-e66d-429c-a23e-d4d0c7bbcc73 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.670589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-048e0781-e66d-429c-a23e-d4d0c7bbcc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.678922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 778/1553] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.680141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fac07-1250-4c61-9585-8423251c53a6 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.680455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fac07-1250-4c61-9585-8423251c53a6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.680724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fac07-1250-4c61-9585-8423251c53a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.688903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-73e44736-a861-4592-86f2-2d59882a846b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.690120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-73e44736-a861-4592-86f2-2d59882a846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.699408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 776/1554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.700630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.700919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.701180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.702650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1f9fac07-1250-4c61-9585-8423251c53a6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.704158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1f9fac07-1250-4c61-9585-8423251c53a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.712617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 779/1555] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.713924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.714363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.714633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.721116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.722327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.730991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 777/1556] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.731811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.735054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.736337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.742099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419793-efa0-485e-82bc-ba08b760eaef None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.748279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 780/1557] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.748279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.748509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be355bf-2ac0-40e2-9be6-3f9151e73ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.748509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be355bf-2ac0-40e2-9be6-3f9151e73ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.748509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be355bf-2ac0-40e2-9be6-3f9151e73ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.748509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be355bf-2ac0-40e2-9be6-3f9151e73ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.749042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be355bf-2ac0-40e2-9be6-3f9151e73ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.750666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be355bf-2ac0-40e2-9be6-3f9151e73ec2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.758007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcb39f-40e6-4026-8cb5-a7811996b6b1 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.760917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 778/1558] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:17.762142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.762465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.762748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.765622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79170a-7f75-49a9-a4dc-570d52c40118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.766268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79170a-7f75-49a9-a4dc-570d52c40118 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.766536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79170a-7f75-49a9-a4dc-570d52c40118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.766805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79170a-7f75-49a9-a4dc-570d52c40118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.767460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79170a-7f75-49a9-a4dc-570d52c40118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.769187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79170a-7f75-49a9-a4dc-570d52c40118 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.779803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 781/1559] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:17.783078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1618d4-eea0-4167-aef5-44bd0498d0be None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.783078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1618d4-eea0-4167-aef5-44bd0498d0be None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.783078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1618d4-eea0-4167-aef5-44bd0498d0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.783359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.783591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.792799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 779/1560] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.793592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.800594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-df1618d4-eea0-4167-aef5-44bd0498d0be None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.801789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-df1618d4-eea0-4167-aef5-44bd0498d0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.802872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214eaf2-ff2e-488c-a9a2-77903ec484d3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.807532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.808125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.808405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.808678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.809327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.809645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 782/1561] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.809792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.810840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.811124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.811249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532dab-c4da-460b-a4c1-fc3ee41556e3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.811419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.824012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 780/1562] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:17.825317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.825627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.825904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.836980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.838360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.844897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:17.846066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:17.846676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 783/1563] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.847475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.857182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b835a-550f-4b80-8ead-4782e1b99d9a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.861285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 781/1564] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:17.861285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.861832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.862467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.862789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.863129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.863731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.864648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.865746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.868076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d9501-8ee6-4f1f-859d-4487844e0957 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.872977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.873603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.873881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.874179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.874858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.876444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.877280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 784/1565] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:17.878073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.887523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 782/1566] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:17.888356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.890708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163184c-562b-4208-8ba1-c02385059ace admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.895780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.896379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.896662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.896935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.897585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.898121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d375ba-6add-4b42-9a52-10950d9038e8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.899180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.903186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.903774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.904052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.904329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.904970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.905270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.906885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.909401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 785/1567] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:17.910288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.919809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55c393-0356-4677-92ca-a51fbd78c8d3 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.929932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 783/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:17.929932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.930222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.930222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.930222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.930222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.930598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.930598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.930598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.933145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea0f8c-973b-42e1-a06c-969d9e13e1c5 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.937945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.938558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.938837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.939104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.946644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 786/1569] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:17.946816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.946913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.947006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.947105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:17.950870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed72175-b428-4e8d-9e19-b0fc617822f5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:17.956627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:17.956797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:17.957294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:17.957688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:17.958135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:17.958960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:17.959174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:17.961008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:17.972307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 784/1570] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:17.974383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.984782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 787/1571] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:17.984782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:17.988526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105853d1-c14f-4153-841f-9c75704df67f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.040283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01e5d3-d7ae-4a8b-b4b6-da1823fe8854 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.045224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.045841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.046171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.046450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.046787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.047115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:18.047407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.047509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.047676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.047957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.048573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.049027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.050674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.061839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 785/1572] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:18.062678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.073702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35c164-8624-4ed8-90eb-d2137ff28851 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.078473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.079138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.079418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.082999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 788/1573] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:17 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.083159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] PATH_INFO: `/v3/projects/b2a35ca5f52e4611b1688b02e221c796/users/d3e4aa96469e4713b2cf1870023e325b/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.083271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.094285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a35ca5f52e4611b1688b02e221c796, user_id=d3e4aa96469e4713b2cf1870023e325b, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.095332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:40:18.095332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:40:18.095332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:40:18.097336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.112830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 786/1574] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:18 2022] PUT /identity/v3/projects/b2a35ca5f52e4611b1688b02e221c796/users/d3e4aa96469e4713b2cf1870023e325b/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:18.113531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.147406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffde7c-d0c0-434e-ad4a-b539796d8a0d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.175475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c50b3-c7af-4753-a30a-04cee6cfff58 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.181694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.182432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.182919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.183435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.184359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.186320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.203400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.204738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.204738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.204738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] PATH_INFO: `/v3/projects/88f96c9ae9b44b91a4db2187160a5688/users/113d48eb42524e62bfc95606f93f3736/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.214653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 787/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.215460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.220208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f96c9ae9b44b91a4db2187160a5688, user_id=113d48eb42524e62bfc95606f93f3736, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.222944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.236190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 789/1576] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:18 2022] PUT /identity/v3/projects/88f96c9ae9b44b91a4db2187160a5688/users/113d48eb42524e62bfc95606f93f3736/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:18.237029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.245294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:18.262654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:18.278300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c4007-0d21-482a-95ab-7defe8cb022b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.310108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268ffc2-423b-4a9c-b989-ad5b9c7ab635 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.314827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.315437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.315713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.316015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.316646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.318654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.334654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.335265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.335532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.335802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.336431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.338380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.349840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 790/1577] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.349840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.358837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfc7af-3edc-4dea-a726-c92525b9768f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.363913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.364458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.364736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.365006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.365632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.367555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.368379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 788/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.369185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.398207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 791/1579] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.398207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.432440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb661ed-8901-4c65-a8d5-82083572953a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.464728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1705ecf-dc42-417b-9acf-9f7fcac5fe1a admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.469297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d0c92-2463-4776-a444-3528711c3257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.469907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d0c92-2463-4776-a444-3528711c3257 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.470215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d0c92-2463-4776-a444-3528711c3257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.470487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d0c92-2463-4776-a444-3528711c3257 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.471107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d0c92-2463-4776-a444-3528711c3257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.473009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d0c92-2463-4776-a444-3528711c3257 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.488310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.488963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.489239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.489505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.490135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.492069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.494681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 792/1580] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.495921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3749-41df-47fc-95b1-385fb12e3417 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.496205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3749-41df-47fc-95b1-385fb12e3417 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.496603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3749-41df-47fc-95b1-385fb12e3417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.521520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 789/1581] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.522368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.526526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7a4e3749-41df-47fc-95b1-385fb12e3417 None None] MFA Rules not processed for user `d3e4aa96469e4713b2cf1870023e325b`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:18.527792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7a4e3749-41df-47fc-95b1-385fb12e3417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:18.536055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:18.586653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:18.586889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0080cf-6594-4ed4-bf66-efe8c32e584d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.591819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.592460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.592725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.592992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.593654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.595764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.598736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 793/1582] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:18.600028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.600311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.600689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.619000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 790/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.619600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.620151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:18.621363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:18.629900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 794/1584] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:18.630709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.631507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f1d0-9695-4ea3-a817-2e2fcc43743e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.636450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.637060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.637335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.637606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.638263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.639866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.641192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf901bd4-2dbb-4012-95e3-9021523c282e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.646742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.647372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.647655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.647929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.648560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.650141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 791/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.650539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.653867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.664224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3f97-e9f8-4958-a971-5f9edc695dbc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.669345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.669941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.670260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.670530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.671143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.673031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.673449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 795/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.674037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.683917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a6bc9-aae3-40de-9326-8a26d127cc5d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.688888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.689488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.689769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.690306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.690707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.692362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.695276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 792/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 282 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.696081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.702825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 796/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.703664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.711209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f081d-a3dd-4601-b99a-43afb7acb071 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.713545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f7cf-7b54-4140-b590-8e0a1cb4e492 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.716464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.717079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.717357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.717629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.718308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.718523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.719127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.719406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.719676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.720222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.720336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.722301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.741947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 793/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.742888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.751754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 797/1590] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.752711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68624f-1ea1-48b2-9351-a99efb592b35 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.755175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.809553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa91d-ecab-4f86-bd10-cdb0138cd730 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.810495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.811123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.811403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.811673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.812326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.813229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.813833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.813944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.814109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.814408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.823514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.823675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 794/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.824350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.824940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.834216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eded5-afbc-415c-93e3-d9a84ef39151 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.835113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 798/1592] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:18.835896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.840110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.840723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.841004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.841274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.841917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.843574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.845894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae261-098b-42e1-b922-46254251e816 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.851298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.852182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.853027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.853489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.854496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.856406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.856572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 795/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.857398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.879468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92484f35-fd7f-401f-922e-8bb5388c4551 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.879610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 799/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.879610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.884560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.885578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.885578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.885780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.886636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.888229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.889464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602c150-f098-4513-9228-6519bfb17286 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.894710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.895325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.895605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.895882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.896520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.898114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.899593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 796/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.900417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.911194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13cef4-69ee-4489-828b-0ae1b7cd431d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.915552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 800/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.915552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.920259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.921400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.921859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.922235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.923000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.925016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.927395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb14c14-cbec-498e-8764-66921aeb03db admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.933097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.933739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.934017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.934326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.934994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.936864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.943448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 797/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.944664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.956640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d174-fc7c-4184-a67c-d3ce6851bb0a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.961413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.965251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.965523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 801/1598] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:18 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:18.965633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.965633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.965849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.965939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.965939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.977238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 798/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:18.978783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df7dcf-2f9e-475d-9876-6ad5b60b1fa0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.978930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:18.984083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.984825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.985298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.985646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] PATH_INFO: `/v3/projects/467b42c2bec74fcbbf90d007a6e00096/users/2aaa8a7f386e4ccc9ab5c356cdf1704f/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.989520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eea6e0-86e8-4812-9a7a-194070a745b8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:18.994103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:18.994810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:18.995158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:18.995494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:18.996141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:18.997730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:18.999346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:19.003197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] RBAC: Authorizing `identity:create_grant(project_id=467b42c2bec74fcbbf90d007a6e00096, user_id=2aaa8a7f386e4ccc9ab5c356cdf1704f, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.006070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.010304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 799/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:19.011203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.023249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 802/1601] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:18 2022] PUT /identity/v3/projects/467b42c2bec74fcbbf90d007a6e00096/users/2aaa8a7f386e4ccc9ab5c356cdf1704f/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.024055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.028300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bead07-ba81-4354-bb4c-0ff4ff7b95b9 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.072021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:19.099585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247425-b9bd-4111-923c-b7e30f8d270d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.101343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.102069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.102475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.103223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.103975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.104710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.105513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.105795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.105982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.106098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] PATH_INFO: `/v3/projects/a23d1f70872f41f6ae52f99f96260a86/users/0f3f5613d232492d85821b87a26a8859/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.117693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 800/1602] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:19.118745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.124247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23d1f70872f41f6ae52f99f96260a86, user_id=0f3f5613d232492d85821b87a26a8859, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.127174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.130115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72067090-100c-4557-8d6e-c7a0860364e5 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.135035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.135720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.136144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.136491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.137213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.138889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.143411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 803/1603] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:19 2022] PUT /identity/v3/projects/a23d1f70872f41f6ae52f99f96260a86/users/0f3f5613d232492d85821b87a26a8859/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.144093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.162299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 801/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:19.162299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.176121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada7507-4936-4235-b5ae-e458a8e8da6e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.244182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523b273-aa55-4cad-9796-054616b512bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.244859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523b273-aa55-4cad-9796-054616b512bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.245135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523b273-aa55-4cad-9796-054616b512bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.245411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523b273-aa55-4cad-9796-054616b512bb admin admin] PATH_INFO: `/v3/projects/c6566aa21ff04589b57219c3be0baebb/users/03ce5f7be7ff454a920a0771fc37fd9a/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.247343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b06a8-5600-43a8-aedc-c0edda4a41cf admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.252408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.253129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.253640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.253975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.254790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.256505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.262199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523b273-aa55-4cad-9796-054616b512bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6566aa21ff04589b57219c3be0baebb, user_id=03ce5f7be7ff454a920a0771fc37fd9a, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.265537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523b273-aa55-4cad-9796-054616b512bb admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.267080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 804/1605] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:19.268006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.279745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e59e4b-60cc-4c83-9614-abac9ff8094f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.283267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 802/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:19 2022] PUT /identity/v3/projects/c6566aa21ff04589b57219c3be0baebb/users/03ce5f7be7ff454a920a0771fc37fd9a/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.285626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc56ae2-fc2d-451a-a3b3-7b43c55292e3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.286253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc56ae2-fc2d-451a-a3b3-7b43c55292e3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.286816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc56ae2-fc2d-451a-a3b3-7b43c55292e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.312673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0fc56ae2-fc2d-451a-a3b3-7b43c55292e3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:19.315721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0fc56ae2-fc2d-451a-a3b3-7b43c55292e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:19.340463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.341128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.341411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.341686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] PATH_INFO: `/v3/projects/8ce26c854fc24cadb2be5ebd441b8f3a/users/43232b9b191047f9a3e6ebab7ad36f02/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.358987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce26c854fc24cadb2be5ebd441b8f3a, user_id=43232b9b191047f9a3e6ebab7ad36f02, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.361637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.377267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 805/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:19 2022] PUT /identity/v3/projects/8ce26c854fc24cadb2be5ebd441b8f3a/users/43232b9b191047f9a3e6ebab7ad36f02/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.378172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.385869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 803/1608] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:19.388487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.389138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.389700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.401819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:19.428511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] MFA Rules not processed for user `0f3f5613d232492d85821b87a26a8859`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:19.430073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:19.432665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:19.460608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5e56a-1109-42f6-bb3a-1cd6e32106a5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.465593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.466660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.466660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.466898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] PATH_INFO: `/v3/projects/044a827b3b044382914b3e5471c39085/users/545d40acc6464704a894cb083ca20bb0/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.484479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044a827b3b044382914b3e5471c39085, user_id=545d40acc6464704a894cb083ca20bb0, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.487255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.502919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 806/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:19 2022] PUT /identity/v3/projects/044a827b3b044382914b3e5471c39085/users/545d40acc6464704a894cb083ca20bb0/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.511187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.514206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 804/1610] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:19.514206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.527575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd4ca6-8635-4c58-aa6f-51c64838fcd7 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.615602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bac5ee-3555-4e60-ad99-1b4b3b21da31 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.620417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d176b-08ca-42f6-8adb-e9fea2881578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.621099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d176b-08ca-42f6-8adb-e9fea2881578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.621380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d176b-08ca-42f6-8adb-e9fea2881578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.621680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d176b-08ca-42f6-8adb-e9fea2881578 admin admin] PATH_INFO: `/v3/projects/65aab6ab3ce94cbd93916edf9c3eedc0/users/192a8abd4c7a4535a42bea7648535dc3/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.623707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b3e76-499f-4cbd-b3ec-78649abaf826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.624751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b3e76-499f-4cbd-b3ec-78649abaf826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.625041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b3e76-499f-4cbd-b3ec-78649abaf826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.625502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b3e76-499f-4cbd-b3ec-78649abaf826 admin admin] PATH_INFO: `/v3/projects/820a7705079844eda2eb6d8a46315de0/users/647212950ae54aeba4a96238bd2d7e10/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.640088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d176b-08ca-42f6-8adb-e9fea2881578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65aab6ab3ce94cbd93916edf9c3eedc0, user_id=192a8abd4c7a4535a42bea7648535dc3, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.643073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d176b-08ca-42f6-8adb-e9fea2881578 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.653961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b3e76-499f-4cbd-b3ec-78649abaf826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820a7705079844eda2eb6d8a46315de0, user_id=647212950ae54aeba4a96238bd2d7e10, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.657573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b3e76-499f-4cbd-b3ec-78649abaf826 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.665727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 807/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:19 2022] PUT /identity/v3/projects/65aab6ab3ce94cbd93916edf9c3eedc0/users/192a8abd4c7a4535a42bea7648535dc3/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.667070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e2c4-94b3-4348-a9c8-58dd12780b99 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.667590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e2c4-94b3-4348-a9c8-58dd12780b99 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.667956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e2c4-94b3-4348-a9c8-58dd12780b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.687846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 805/1612] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:19 2022] PUT /identity/v3/projects/820a7705079844eda2eb6d8a46315de0/users/647212950ae54aeba4a96238bd2d7e10/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:19.689134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.689421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.689693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.709063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0c68e2c4-94b3-4348-a9c8-58dd12780b99 None None] MFA Rules not processed for user `03ce5f7be7ff454a920a0771fc37fd9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:19.710580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:19.711273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0c68e2c4-94b3-4348-a9c8-58dd12780b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:19.711769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:19.785223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 806/1613] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:19.786170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.800432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 808/1614] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:19.802431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dbe8d-98c3-4bd8-8b76-29778e1c4f64 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.802725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dbe8d-98c3-4bd8-8b76-29778e1c4f64 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.803003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dbe8d-98c3-4bd8-8b76-29778e1c4f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.824091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:19.864344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5a3dbe8d-98c3-4bd8-8b76-29778e1c4f64 None None] MFA Rules not processed for user `545d40acc6464704a894cb083ca20bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:19.866564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5a3dbe8d-98c3-4bd8-8b76-29778e1c4f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:19.880289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd000bf-9cdc-45c3-84e7-bee8161a4a7c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.884308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.884939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.885217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.885616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.895675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:19.903020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.904498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.922029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 807/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:19.923400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:19.936962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3851-f061-48a4-a3d1-c9f0e710f708 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:19.945846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:19.947472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.947973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.948950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.950887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:19.954447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:19.956641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 809/1616] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:19.957949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:19.958388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:19.958661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:19.982973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:19.984736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:19.994732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 810/1617] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:19.998599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.009776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 808/1618] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:19 2022] POST /identity/v3/projects => generated 403 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:20.009776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.009776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:20.010189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130395e9-8f77-4e5a-b3aa-7aa63a2c3388 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.093207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a12da-ed1c-4423-8138-27900bded743 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.099102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.099983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.100614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.101067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.108517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.109171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.109898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.110217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.111020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.112787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.128399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 811/1619] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:20.129660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.190978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.192965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.208658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 809/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:20.221536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.239780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212009f2-3b1d-401e-9448-42d60aa916d0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.246861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.247909 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.248475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.248679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.249555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52c1e4-8a22-428b-b09b-24eed7d08a90 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.259787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.260477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.260891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.261092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.261805 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.265440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.337226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 810/1621] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:20 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:20.343514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.377630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.379561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.393571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 812/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:20.394952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.409681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd1315-e1b7-4519-a175-60f84155ac84 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.419626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:20.471227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:20.531631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911648fa-077c-4239-9f3e-b427142f0cad admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.535488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.536506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.536726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.537051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.538053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.540454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.540619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.541086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.541362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.541633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.563826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.565550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.573089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 813/1623] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:20 2022] POST /identity/v3/users => generated 308 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:20.574232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.590123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460461fa-2693-431e-8b90-9f3460d10758 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.591524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 811/1624] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:20.593739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.596505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.597391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.597861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.598139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] PATH_INFO: `/v3/projects/820a7705079844eda2eb6d8a46315de0/users/647212950ae54aeba4a96238bd2d7e10/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.603156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820a7705079844eda2eb6d8a46315de0, user_id=647212950ae54aeba4a96238bd2d7e10, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.606647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.616889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21aa79-8177-49ee-8010-0296e15920fb service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.624073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 814/1625] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:20 2022] PUT /identity/v3/projects/820a7705079844eda2eb6d8a46315de0/users/647212950ae54aeba4a96238bd2d7e10/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:20.625108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.738621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905fb1-2f43-47a3-9d93-a25b13feb433 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.745549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fecf2-be8f-4821-a1c4-e79b60fb93e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.746794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fecf2-be8f-4821-a1c4-e79b60fb93e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.747083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fecf2-be8f-4821-a1c4-e79b60fb93e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.747622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fecf2-be8f-4821-a1c4-e79b60fb93e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.749137 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fecf2-be8f-4821-a1c4-e79b60fb93e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.751876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fecf2-be8f-4821-a1c4-e79b60fb93e5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.823642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 815/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:20.823642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.823642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.823642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.825196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.825196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.825196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.825196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.825634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.825634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.825777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] MFA Rules not processed for user `647212950ae54aeba4a96238bd2d7e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:20.825777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:20.849632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 812/1627] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:20 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:20.855970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.877529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197450-bd15-4ea8-9274-78f62dcc090e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:20.883068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:20.884864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:20.885152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:20.885602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] PATH_INFO: `/v3/projects/200788ab45724a02842f500c96c34fb0/users/867d85d84bf640ccb7e95e258d9401b5/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:20.904093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200788ab45724a02842f500c96c34fb0, user_id=867d85d84bf640ccb7e95e258d9401b5, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:20.907067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:20.925861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 813/1628] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:20 2022] PUT /identity/v3/projects/200788ab45724a02842f500c96c34fb0/users/867d85d84bf640ccb7e95e258d9401b5/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:20.927015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.953044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 816/1629] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:20.953044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:20.973251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:20.986845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bffc5-c79f-45df-be29-f8c20e2ad61f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.041670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91bd20-e95a-479d-9bd6-a8e5c4b81cf6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.054355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.055600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.055600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.055892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.056747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.067406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.079561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.080342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.080701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.081042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.081847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.084645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.103649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 817/1630] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:21.104728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:21.122434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 814/1631] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:21.125364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:21.142381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fa75e-8882-4480-9233-d7ecb081aca8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.150710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.151539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.151909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.152291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] PATH_INFO: `/v3/projects/200788ab45724a02842f500c96c34fb0/users/867d85d84bf640ccb7e95e258d9401b5/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.154570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200788ab45724a02842f500c96c34fb0, user_id=867d85d84bf640ccb7e95e258d9401b5, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.160923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.184920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d412b5e-62b9-4776-9816-555a16371eb9 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.191968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 815/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:21 2022] PUT /identity/v3/projects/200788ab45724a02842f500c96c34fb0/users/867d85d84bf640ccb7e95e258d9401b5/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:21.192920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:21.244390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78309b5-3055-487c-a3ce-1947535b67fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.244975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78309b5-3055-487c-a3ce-1947535b67fb service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.245634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78309b5-3055-487c-a3ce-1947535b67fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.246397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78309b5-3055-487c-a3ce-1947535b67fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.251198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:21.384028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78309b5-3055-487c-a3ce-1947535b67fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.385758 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78309b5-3055-487c-a3ce-1947535b67fb service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.400782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa32bee-3329-485b-b6be-8c77996dca90 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.403477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 818/1633] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:21.407243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.407531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.407805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.427116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.427831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.428111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.428982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] PATH_INFO: `/v3/projects/adba80ced89d4d5aa281fa8a897be828/users/813340069f114c4f94b1fd9e0a9da503/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.434960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:21.467870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adba80ced89d4d5aa281fa8a897be828, user_id=813340069f114c4f94b1fd9e0a9da503, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.470609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.472113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] MFA Rules not processed for user `867d85d84bf640ccb7e95e258d9401b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:21.476316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:21.513840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 816/1634] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:21 2022] PUT /identity/v3/projects/adba80ced89d4d5aa281fa8a897be828/users/813340069f114c4f94b1fd9e0a9da503/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:21.517741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:21.715930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 819/1635] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:21.730333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:21.734413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9c6f-f8cf-4c04-bdaf-d6baf7d94ed5 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.743446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19deafb-f784-4d09-9739-0068a6550991 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.756767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.757425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.761992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.762302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.763008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.771151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.810711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 817/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:21.811754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:21.822437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.823176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.823463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.823947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.845209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8698230-9f2f-413e-a689-ac7cf5264ef8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:21.858042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67597f6-4d03-4322-a711-d39da8819087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:21.858837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67597f6-4d03-4322-a711-d39da8819087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.859120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67597f6-4d03-4322-a711-d39da8819087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.859397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67597f6-4d03-4322-a711-d39da8819087 admin admin] PATH_INFO: `/v3/projects/adba80ced89d4d5aa281fa8a897be828/users/813340069f114c4f94b1fd9e0a9da503/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.863675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67597f6-4d03-4322-a711-d39da8819087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adba80ced89d4d5aa281fa8a897be828, user_id=813340069f114c4f94b1fd9e0a9da503, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:21.871571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67597f6-4d03-4322-a711-d39da8819087 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:21.898191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 818/1637] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:21 2022] PUT /identity/v3/projects/adba80ced89d4d5aa281fa8a897be828/users/813340069f114c4f94b1fd9e0a9da503/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:21.908282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:21.908403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:21.908679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:21.967101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] MFA Rules not processed for user `813340069f114c4f94b1fd9e0a9da503`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:21.969063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:22.046684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:22.051290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:22.099018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 820/1638] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:21 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:22.128579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 819/1639] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:22.201364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:22.220374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19be0b6-fc5a-4a37-a82a-142aabca1b6a service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:22.303107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:22.339477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:22.340276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:22.340569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:22.340845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:22.366497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:22.367873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:22.390626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 821/1640] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:22.600368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:22.610296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:22.636782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59265bd-6512-415d-8805-679060be8366 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:22.642694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:22.657788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2396f1-a00c-4a89-aabe-56272d0de846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:22.658076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2396f1-a00c-4a89-aabe-56272d0de846 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:22.658218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2396f1-a00c-4a89-aabe-56272d0de846 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:22.658466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2396f1-a00c-4a89-aabe-56272d0de846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:22.828170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a705ffc-bab6-4bc8-a9c8-3e708c31a41c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:22.842733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2396f1-a00c-4a89-aabe-56272d0de846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:22.844886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2396f1-a00c-4a89-aabe-56272d0de846 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:22.853210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:22.853933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:22.854251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:22.854707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] PATH_INFO: `/v3/users/0f3f5613d232492d85821b87a26a8859` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:22.864145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f3f5613d232492d85821b87a26a8859)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:22.865213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: WARNING py.warnings [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:40:22.865213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: warnings.warn(msg) Aug 09 02:40:22.865213 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: Aug 09 02:40:22.866472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:22.877121 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 822/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:23.052322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f3f5613d232492d85821b87a26a8859'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:23.074420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f3f5613d232492d85821b87a26a8859'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:23.076367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f3f5613d232492d85821b87a26a8859'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:23.108011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f3f5613d232492d85821b87a26a8859'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:23.154006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3ce4a8b7-366a-4bd6-8008-dd7db7ec8668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f3f5613d232492d85821b87a26a8859'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:23.167013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 820/1642] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:22 2022] DELETE /identity/v3/users/0f3f5613d232492d85821b87a26a8859 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:23.168656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:23.168951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:23.169232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:23.213163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:23.215115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:23.421199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9b233-31ce-4803-bf18-3c00415d7b33 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:23.421561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9b233-31ce-4803-bf18-3c00415d7b33 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:23.421905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9b233-31ce-4803-bf18-3c00415d7b33 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:23.424040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 821/1643] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 9 02:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:23.454938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e001c-5ffb-4b57-82e6-d5795af0436d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:23.454938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e001c-5ffb-4b57-82e6-d5795af0436d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:23.455211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e001c-5ffb-4b57-82e6-d5795af0436d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:23.457740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 823/1644] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:23.496466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:23.498270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:23.571425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-3e3e001c-5ffb-4b57-82e6-d5795af0436d None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:23.572892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3e3e001c-5ffb-4b57-82e6-d5795af0436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:23.725944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b8088-a763-4649-a07f-4b6484f0cf90 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:23.738893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:23.741045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:23.744193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:23.745360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:23.747582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 822/1645] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 9 02:40:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:23.749184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e65da-3810-4275-a6c0-2d36a9994ad2 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:23.749490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e65da-3810-4275-a6c0-2d36a9994ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:23.749778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e65da-3810-4275-a6c0-2d36a9994ad2 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:23.751599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 823/1646] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 9 02:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:23.767027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:23.767359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:23.767730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:23.789728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:23.794016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:23.797305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:23.808883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:23.811647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:23.825478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 824/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:23.854538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 824/1648] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:40:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:23.873320 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:23.881853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:23.969434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:24.025916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b60e-27ae-4b22-afee-ae7aab908672 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:24.045036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a213b-2349-475d-831a-1eaa423940ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:24.091527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a213b-2349-475d-831a-1eaa423940ff service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:24.091527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a213b-2349-475d-831a-1eaa423940ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:24.091527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a213b-2349-475d-831a-1eaa423940ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:24.098708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a213b-2349-475d-831a-1eaa423940ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:24.106247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a213b-2349-475d-831a-1eaa423940ff service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:24.147932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 825/1649] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:25.348424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:25.350661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:25.390985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710853d8-5e2d-462d-bd46-758f7ef6c243 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:25.415107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:25.416151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:25.416621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:25.417053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] PATH_INFO: `/v3/users/d3e4aa96469e4713b2cf1870023e325b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:25.422008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e4aa96469e4713b2cf1870023e325b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:25.428218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:25.584124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e4aa96469e4713b2cf1870023e325b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:25.612784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e4aa96469e4713b2cf1870023e325b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:25.613887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e4aa96469e4713b2cf1870023e325b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:25.631847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e4aa96469e4713b2cf1870023e325b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:25.669523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ea2a005f-4d1e-4a5e-a1dd-73da1cd64914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e4aa96469e4713b2cf1870023e325b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:25.698099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 825/1650] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:25 2022] DELETE /identity/v3/users/d3e4aa96469e4713b2cf1870023e325b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:25.702727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:25.703225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:25.703309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:25.742266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:25.744031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:25.896267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 826/1651] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:25.997254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccce9e-e985-4849-9474-8e40c82bffcd None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:25.997636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccce9e-e985-4849-9474-8e40c82bffcd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:25.998023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccce9e-e985-4849-9474-8e40c82bffcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.047514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:26.049353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.060917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-68ccce9e-e985-4849-9474-8e40c82bffcd None None] MFA Rules not processed for user `113d48eb42524e62bfc95606f93f3736`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:26.062386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-68ccce9e-e985-4849-9474-8e40c82bffcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.165540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f18a65-e0bc-4fad-9df2-12a3bf0faea8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:26.173756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ed2f8-64d4-41db-8eec-64f66068f3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:26.174349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ed2f8-64d4-41db-8eec-64f66068f3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.174560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ed2f8-64d4-41db-8eec-64f66068f3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.174838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ed2f8-64d4-41db-8eec-64f66068f3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.197620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ed2f8-64d4-41db-8eec-64f66068f3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:26.212313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 826/1652] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:26.214955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfbec7-9164-48e3-bdb4-f15e6c64ece4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.214955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfbec7-9164-48e3-bdb4-f15e6c64ece4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.215152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfbec7-9164-48e3-bdb4-f15e6c64ece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.219533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ed2f8-64d4-41db-8eec-64f66068f3e6 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:26.240681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a8dfbec7-9164-48e3-bdb4-f15e6c64ece4 None None] MFA Rules not processed for user `113d48eb42524e62bfc95606f93f3736`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:26.242023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a8dfbec7-9164-48e3-bdb4-f15e6c64ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.242477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 827/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:26.250202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 827/1654] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:26.286188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ca2d4-ab9b-4b64-8e75-cbfc5f2668f2 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.286476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ca2d4-ab9b-4b64-8e75-cbfc5f2668f2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.286780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ca2d4-ab9b-4b64-8e75-cbfc5f2668f2 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.291195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 828/1655] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 9 02:40:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:26.310094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.310524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.310708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.349756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:26.351333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.405831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ca2d4-ab9b-4b64-8e75-cbfc5f2668f2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:26.412449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-bd3ca2d4-ab9b-4b64-8e75-cbfc5f2668f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.459818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ca2d4-ab9b-4b64-8e75-cbfc5f2668f2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:26.464671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:26.465331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.465614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.465891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] PATH_INFO: `/v3/projects/a23d1f70872f41f6ae52f99f96260a86` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.467283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23d1f70872f41f6ae52f99f96260a86)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:26.468708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: WARNING py.warnings [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 02:40:26.468708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: warnings.warn(msg) Aug 09 02:40:26.468708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: Aug 09 02:40:26.470275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:26.511273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:26.574862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 828/1656] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 9 02:40:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:26.583379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:26.600106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d114334-1e77-4535-b78d-9b805ff76391 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:26.627742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23d1f70872f41f6ae52f99f96260a86', 'request_id': 'req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:26.656231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23d1f70872f41f6ae52f99f96260a86', 'request_id': 'req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:26.665959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23d1f70872f41f6ae52f99f96260a86', 'request_id': 'req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:26.670228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:26.691219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] The token cache is being invalidate because project a23d1f70872f41f6ae52f99f96260a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:26.691925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:26.692799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:26.696922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 829/1657] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:26 2022] DELETE /identity/v3/projects/a23d1f70872f41f6ae52f99f96260a86 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:26.703060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:26.706075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.814443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:26.817080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.818337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.819186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.826914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:26.878825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6c2de-02a8-4e9e-852f-7c13d937ab2c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:26.898862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:26.899587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:26.899871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:26.900147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] PATH_INFO: `/v3/users/867d85d84bf640ccb7e95e258d9401b5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:26.905286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=867d85d84bf640ccb7e95e258d9401b5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:26.907852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:26.998522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:27.004778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:27.039090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 829/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:27.094062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867d85d84bf640ccb7e95e258d9401b5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.095355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867d85d84bf640ccb7e95e258d9401b5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.134230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867d85d84bf640ccb7e95e258d9401b5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.161676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867d85d84bf640ccb7e95e258d9401b5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.203296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5cd66ca8-b390-4abc-9543-607311ea0157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867d85d84bf640ccb7e95e258d9401b5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.214680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 830/1659] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:26 2022] DELETE /identity/v3/users/867d85d84bf640ccb7e95e258d9401b5 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:27.335902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:27.337640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:27.379164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaca5f8-64c2-4032-aa62-9034fc533940 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:27.432539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d48e-7c54-4cd8-9d4d-4e201867c9cf None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:27.432993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d48e-7c54-4cd8-9d4d-4e201867c9cf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:27.433373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d48e-7c54-4cd8-9d4d-4e201867c9cf None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:27.435146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 831/1660] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 9 02:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:27.442750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:27.443426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:27.443831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:27.474947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:27.476386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:27.482626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:27.483806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:27.484089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:27.484363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] PATH_INFO: `/v3/users/545d40acc6464704a894cb083ca20bb0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:27.485578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=545d40acc6464704a894cb083ca20bb0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:27.488185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:27.535933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:27.571982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '545d40acc6464704a894cb083ca20bb0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.593559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '545d40acc6464704a894cb083ca20bb0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.594499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '545d40acc6464704a894cb083ca20bb0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.607170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '545d40acc6464704a894cb083ca20bb0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.626716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 832/1661] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:40:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:27.639434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e4f3c8a4-419f-4112-b88e-d73a3c56dd0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '545d40acc6464704a894cb083ca20bb0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:27.643741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:27.645745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:27.661378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:27.667966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 830/1662] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:27 2022] DELETE /identity/v3/users/545d40acc6464704a894cb083ca20bb0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:27.676954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:27.677240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:27.677519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:27.709248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:27.710595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:27.873787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 831/1663] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:27.874384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:27.876521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:27.892670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ab79c-4fcc-483d-8bfd-a5211a6abc96 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:27.919080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ac598-6312-423b-b2ed-0d5691ed53a6 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:27.930090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:27.930855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:27.931103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:27.931380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] PATH_INFO: `/v3/projects/200788ab45724a02842f500c96c34fb0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:27.932721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=200788ab45724a02842f500c96c34fb0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:27.936604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:27.985226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:27.986035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:27.986766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:27.986994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.047646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200788ab45724a02842f500c96c34fb0', 'request_id': 'req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:28.067881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200788ab45724a02842f500c96c34fb0', 'request_id': 'req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:28.079259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200788ab45724a02842f500c96c34fb0', 'request_id': 'req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:28.086901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.088638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.099030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] The token cache is being invalidate because project 200788ab45724a02842f500c96c34fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:28.099306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:28.104547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bff2ef01-b726-4e99-a634-6b29cbe2ff9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:28.106281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 833/1664] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:28.107156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.108647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:28.117899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 832/1665] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:27 2022] DELETE /identity/v3/projects/200788ab45724a02842f500c96c34fb0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:28.123464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530a57-df24-43b5-824b-c58197232275 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.123761 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530a57-df24-43b5-824b-c58197232275 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.124037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530a57-df24-43b5-824b-c58197232275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.185796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6a530a57-df24-43b5-824b-c58197232275 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:28.191643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a530a57-df24-43b5-824b-c58197232275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:28.257367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:28.272941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:28.275815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbde56-9eac-45e5-8d52-45ca6f70b29e service glance] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:28.375705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:28.376782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.377318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.377596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.380772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:28.421998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 833/1666] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:28.429779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.430094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.430370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.494436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:28.496101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:28.530590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 834/1667] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:28.531692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.543086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.543870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495dad-8027-4923-ae0e-620b074ff03c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:28.544560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.551188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:28.551823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.552100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.552378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.553039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:28.553341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.555579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.575492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 834/1668] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:28.591071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.593905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 835/1669] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:28.603523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9e860-dc23-4a04-aab8-e7162ec1e04b service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:28.611155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:28.611795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.612075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.612344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.612976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.615451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.657986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 835/1670] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:28 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:40:28.663361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.677857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108b5df-8d27-42b0-98b0-98b2a6ce1a70 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:28.773177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:28.773874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.774209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.774483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.775225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.777328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.813646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 836/1671] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:28 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:28.814709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.830900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de0dcb-a696-4b74-8be0-34e95fd3b8f2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:28.844098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:28.844759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.845094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.845376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.846038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.856112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.896082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 836/1672] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:28.901368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.917728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f4c36-687f-436b-9e5d-53821cde0c9f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:28.930044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0971e43-2829-4873-b79f-ecd8989b9e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:28.931164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0971e43-2829-4873-b79f-ecd8989b9e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:28.931592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0971e43-2829-4873-b79f-ecd8989b9e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:28.931957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0971e43-2829-4873-b79f-ecd8989b9e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:28.932716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0971e43-2829-4873-b79f-ecd8989b9e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:28.934547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0971e43-2829-4873-b79f-ecd8989b9e6d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:28.959433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 837/1673] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:28.962387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:28.980274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd0b62-744a-4239-afd5-ef49e5ee9868 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:29.001861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2f322-23ef-41d8-bee3-0b3bb1f33799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:29.001861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2f322-23ef-41d8-bee3-0b3bb1f33799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.001861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2f322-23ef-41d8-bee3-0b3bb1f33799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.001861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2f322-23ef-41d8-bee3-0b3bb1f33799 admin admin] PATH_INFO: `/v3/projects/16722152c8004b609a4d900ba50b3085/users/692335d9e9e64c0f8e1c581ed6048fa9/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.072623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae52eac-4f88-42d0-88c8-9b09b0c50785 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.073280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae52eac-4f88-42d0-88c8-9b09b0c50785 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.074201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae52eac-4f88-42d0-88c8-9b09b0c50785 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.076987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 838/1674] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 9 02:40:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:40:29.079776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2f322-23ef-41d8-bee3-0b3bb1f33799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16722152c8004b609a4d900ba50b3085, user_id=692335d9e9e64c0f8e1c581ed6048fa9, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:29.084904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.085445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.085904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.089690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2f322-23ef-41d8-bee3-0b3bb1f33799 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:29.144820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 837/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:28 2022] PUT /identity/v3/projects/16722152c8004b609a4d900ba50b3085/users/692335d9e9e64c0f8e1c581ed6048fa9/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:29.157633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.158109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.158409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.164264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:29.173108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:29.306716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] MFA Rules not processed for user `692335d9e9e64c0f8e1c581ed6048fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:29.308181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:29.336294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 839/1676] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:40:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 09 02:40:29.348478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:29.353288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:29.401386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:29.423972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:29.488426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 838/1677] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:29.492606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:29.500590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:29.521950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68373c-b02c-4e78-84fc-c9f4208943e4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:29.532453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:29.533197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.533478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.534578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] PATH_INFO: `/v3/projects/044a827b3b044382914b3e5471c39085` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.537466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=044a827b3b044382914b3e5471c39085)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:29.541984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:29.542143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e00da-01e7-4bdc-bf20-242f59dcfe38 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:29.551692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:29.552344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.553711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.554001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.574627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:29.576116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:29.590620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 839/1678] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:29.598224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:29.628868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dabcc-7480-4798-973c-85ef9e209588 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:29.646144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044a827b3b044382914b3e5471c39085', 'request_id': 'req-863d933d-9e3b-4530-836d-235ecad32b9a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:29.667389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044a827b3b044382914b3e5471c39085', 'request_id': 'req-863d933d-9e3b-4530-836d-235ecad32b9a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:29.677975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044a827b3b044382914b3e5471c39085', 'request_id': 'req-863d933d-9e3b-4530-836d-235ecad32b9a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:29.701521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] The token cache is being invalidate because project 044a827b3b044382914b3e5471c39085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:29.703821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:29.704737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:29.710763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 840/1679] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:29 2022] DELETE /identity/v3/projects/044a827b3b044382914b3e5471c39085 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:29.713940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:29.716328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:29.745371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:29.746015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:29.746327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:29.746633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:29.747416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:29.749730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:29.813780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 840/1680] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:29 2022] POST /identity/v3/projects => generated 381 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:29.814532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:29.819440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:29.922234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d933d-9e3b-4530-836d-235ecad32b9a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.008763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77148a26-2fb6-4207-b4f6-2b38c3942c67 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.030133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.037268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.039190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.040073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] PATH_INFO: `/v3/users/813340069f114c4f94b1fd9e0a9da503` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.044242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] RBAC: Authorizing `identity:delete_user(user_id=813340069f114c4f94b1fd9e0a9da503)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:30.053413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:30.138492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.139409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.139741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.143413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] PATH_INFO: `/v3/projects/b2a35ca5f52e4611b1688b02e221c796` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.146812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a35ca5f52e4611b1688b02e221c796)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:30.150546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:30.152688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:30.189742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813340069f114c4f94b1fd9e0a9da503'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.190846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813340069f114c4f94b1fd9e0a9da503'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.208584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813340069f114c4f94b1fd9e0a9da503'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.211387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:30.220681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813340069f114c4f94b1fd9e0a9da503'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.249212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813340069f114c4f94b1fd9e0a9da503'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.263327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a35ca5f52e4611b1688b02e221c796', 'request_id': 'req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.264904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 841/1681] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:29 2022] DELETE /identity/v3/users/813340069f114c4f94b1fd9e0a9da503 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:30.267153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:30.268832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:30.282809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a35ca5f52e4611b1688b02e221c796', 'request_id': 'req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.301286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a35ca5f52e4611b1688b02e221c796', 'request_id': 'req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.332548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] The token cache is being invalidate because project b2a35ca5f52e4611b1688b02e221c796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:30.332858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.334418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.336258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 841/1682] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:29 2022] DELETE /identity/v3/projects/b2a35ca5f52e4611b1688b02e221c796 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:30.340360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:30.342078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:30.475555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1906f01-cf6f-495d-ae06-9c258448045d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.490058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.491374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.492820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.493228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.493953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:30.496665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:30.527603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 842/1683] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:30 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:30.530776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8c49-4e13-4223-9c20-4fde6b62a8a6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.533399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:30.543314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.544239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.545129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.545129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] PATH_INFO: `/v3/users/647212950ae54aeba4a96238bd2d7e10` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.546821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=647212950ae54aeba4a96238bd2d7e10)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:30.550495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:30.551492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:30.588731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290b233-838c-4535-8f09-071f525f5047 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.602400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.603813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.604305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.604976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.606185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:30.609245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:30.628530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 843/1684] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:30.629849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:30.632629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:30.643013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '647212950ae54aeba4a96238bd2d7e10'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.664314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '647212950ae54aeba4a96238bd2d7e10'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.665232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '647212950ae54aeba4a96238bd2d7e10'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.678196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '647212950ae54aeba4a96238bd2d7e10'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.708231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '647212950ae54aeba4a96238bd2d7e10'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:30.723176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 842/1685] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:30 2022] DELETE /identity/v3/users/647212950ae54aeba4a96238bd2d7e10 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:30.724215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:30.728149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:30.751753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1be43-d469-4af1-a1b9-2a5ae02c5b79 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.757609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:30.861089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:30.887399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.888176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.889073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.889073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.911253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:30.953261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7944d01-4559-401b-b88a-731e5ccfbd0b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:30.959062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:30.960791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:30.964586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:30.965806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:30.966369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:30.967801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] PATH_INFO: `/v3/projects/da837c9180b942c4a723ae9c24dc6fb7/users/a708a93803b346acb235375988584c93/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:30.995350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 844/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:30.999878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=da837c9180b942c4a723ae9c24dc6fb7, user_id=a708a93803b346acb235375988584c93, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:31.003449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:31.036420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 843/1687] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:30 2022] PUT /identity/v3/projects/da837c9180b942c4a723ae9c24dc6fb7/users/a708a93803b346acb235375988584c93/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:31.040800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:31.059763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd539f9-0993-4079-a0cb-c1f28c4b8477 service placement] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:31.239012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6a136-0a65-42e0-9156-53e855d4064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:31.239857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a136-0a65-42e0-9156-53e855d4064c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:31.241066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a136-0a65-42e0-9156-53e855d4064c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:31.241352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a136-0a65-42e0-9156-53e855d4064c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:31.242814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a136-0a65-42e0-9156-53e855d4064c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:31.244678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a136-0a65-42e0-9156-53e855d4064c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:31.275171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 845/1688] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:31.279367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:31.299539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ecb6-e7a1-4e68-ab21-34d56cdef0dd admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:31.313703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:31.314453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:31.314735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:31.315001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] PATH_INFO: `/v3/projects/da837c9180b942c4a723ae9c24dc6fb7/users/a708a93803b346acb235375988584c93/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:31.331521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da837c9180b942c4a723ae9c24dc6fb7, user_id=a708a93803b346acb235375988584c93, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:31.347318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:31.382023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 844/1689] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:31 2022] PUT /identity/v3/projects/da837c9180b942c4a723ae9c24dc6fb7/users/a708a93803b346acb235375988584c93/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:31.393086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:31.393380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:31.393659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:31.483249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] MFA Rules not processed for user `a708a93803b346acb235375988584c93`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:31.486483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:31.734302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 846/1690] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:31.749308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:31.751322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:31.818643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:31.820272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:31.980652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7b11-8f1b-40f4-b42a-ff6661d679f0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:31.997983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.000888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.001186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.001461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.025661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.028265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.037109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce78f8-dd23-4fac-805c-b3417e50d0c7 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.049269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.049959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.050308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.050586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] PATH_INFO: `/v3/projects/adba80ced89d4d5aa281fa8a897be828` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.056954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adba80ced89d4d5aa281fa8a897be828)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.060019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 845/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:32.061220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:32.063108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.065974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.087709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:32.099916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36b85e-59ea-4173-8aa4-1a7fa6b004a6 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.110173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.112475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.112827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.113151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] PATH_INFO: `/v3/projects/820a7705079844eda2eb6d8a46315de0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.122218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=820a7705079844eda2eb6d8a46315de0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.131780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.232326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adba80ced89d4d5aa281fa8a897be828', 'request_id': 'req-d766f726-e197-438e-896c-b15ec9f384b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.271835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adba80ced89d4d5aa281fa8a897be828', 'request_id': 'req-d766f726-e197-438e-896c-b15ec9f384b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.286209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820a7705079844eda2eb6d8a46315de0', 'request_id': 'req-58c7184e-b2d6-4987-8db5-47d2c128a680'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.289426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adba80ced89d4d5aa281fa8a897be828', 'request_id': 'req-d766f726-e197-438e-896c-b15ec9f384b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.312941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820a7705079844eda2eb6d8a46315de0', 'request_id': 'req-58c7184e-b2d6-4987-8db5-47d2c128a680'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.328914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820a7705079844eda2eb6d8a46315de0', 'request_id': 'req-58c7184e-b2d6-4987-8db5-47d2c128a680'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.329116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] The token cache is being invalidate because project adba80ced89d4d5aa281fa8a897be828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:32.329513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.330476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.332790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 847/1692] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:31 2022] DELETE /identity/v3/projects/adba80ced89d4d5aa281fa8a897be828 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:32.334203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:32.338325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.347716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] The token cache is being invalidate because project 820a7705079844eda2eb6d8a46315de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:32.348304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.349352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-58c7184e-b2d6-4987-8db5-47d2c128a680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:32.352135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 846/1693] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:32 2022] DELETE /identity/v3/projects/820a7705079844eda2eb6d8a46315de0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:32.354731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1656906-1229-4ba6-a8db-117db7fd3d26 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.355259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1656906-1229-4ba6-a8db-117db7fd3d26 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.356020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1656906-1229-4ba6-a8db-117db7fd3d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.368953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:32.395267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c1656906-1229-4ba6-a8db-117db7fd3d26 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:32.397882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c1656906-1229-4ba6-a8db-117db7fd3d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.443467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766f726-e197-438e-896c-b15ec9f384b6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.566802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 847/1694] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:32.570386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f217-8491-41fe-8f91-2cba56cef4bf None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.571218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f217-8491-41fe-8f91-2cba56cef4bf None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.571534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f217-8491-41fe-8f91-2cba56cef4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.596307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.597875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.598723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.602471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.613272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.634044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1cb9f217-8491-41fe-8f91-2cba56cef4bf None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:32.638492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1cb9f217-8491-41fe-8f91-2cba56cef4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.664839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 848/1695] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:32.667721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e1384-c85b-483e-8e96-b2e0e6b6bd9f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.669091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e1384-c85b-483e-8e96-b2e0e6b6bd9f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.669454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e1384-c85b-483e-8e96-b2e0e6b6bd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.723388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-179e1384-c85b-483e-8e96-b2e0e6b6bd9f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:32.724986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-179e1384-c85b-483e-8e96-b2e0e6b6bd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.748160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 849/1696] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:32.751323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.751878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.751878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.809642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:32.813227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:32.816879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.822355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.828142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 850/1697] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:32.829790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:32.851505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a7b89-dd8c-44b8-9b53-14176d5b1010 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.856686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 848/1698] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:32.857648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:32.862735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.864021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.865689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.866763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.867673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:32.869078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.873855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.881492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce262c-2eec-4861-924f-da00ffa24978 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.892937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.895095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.895716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.897064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.897834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:32.898184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.905708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 851/1699] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:32.906952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:32.911268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.913244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:32.932299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00efe0e-3f43-4f86-bc21-a9aa37292583 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.946598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 849/1700] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:32.947862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:32.950709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.951355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.951692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.951930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.952576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:32.955222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:32.976214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce99619-f580-4a1a-a8cf-c7ba90817f64 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:32.992427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:32.993321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:32.993769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:32.995991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:32.996903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.006446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.020999 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 852/1701] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:32 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:33.026748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.099103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 850/1702] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:32 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:33.104832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.201597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c397b-da81-44b3-bfbc-2c0582b28833 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.264389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24296aea-831c-4bb8-951c-b4226b68f937 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.275510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.276213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.276504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.276797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.277445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.287111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.332354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:33.343488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 851/1703] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:33 2022] POST /identity/v3/users => generated 284 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:40:33.346423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.349686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.350384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.351556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.351831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.355436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.361109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.373470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d9d29-44a8-46dd-8ae5-6e7eb25ef651 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.389883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.390563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.390841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.391123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.391821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.393721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.435517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 853/1704] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:33 2022] POST /identity/v3/users => generated 290 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:33.437207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.440757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 852/1705] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:33.441869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.448915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc53297-7bd8-4654-8d61-dc939ae6e9e6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.455126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.455807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.456089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.456392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.457179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.459083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.473276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 854/1706] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:33.478615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.494247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf3cbc-b729-4a83-958c-526ded899a05 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.500626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1e843-cd3a-4449-805d-47552510037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.501575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1e843-cd3a-4449-805d-47552510037f admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.501961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1e843-cd3a-4449-805d-47552510037f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.502428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1e843-cd3a-4449-805d-47552510037f admin admin] PATH_INFO: `/v3/projects/0dc5596725f34e7781ec7b1ae4c7fbb8/users/70c579b22f7b4160bec240b976b64526/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.503464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab992-bdf9-4382-a4ee-15d4ad8a0776 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.535376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1e843-cd3a-4449-805d-47552510037f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc5596725f34e7781ec7b1ae4c7fbb8, user_id=70c579b22f7b4160bec240b976b64526, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.536119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.536979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.537484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.537832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.538590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.541857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1e843-cd3a-4449-805d-47552510037f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.543711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.584565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 855/1707] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:33 2022] PUT /identity/v3/projects/0dc5596725f34e7781ec7b1ae4c7fbb8/users/70c579b22f7b4160bec240b976b64526/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:33.586827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.587117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.587715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.597134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 853/1708] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:33.601200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.620785 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa50ed-8f70-45c5-b90c-e9f3203c00ce admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.657446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] MFA Rules not processed for user `70c579b22f7b4160bec240b976b64526`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:33.659311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:33.765712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.766327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.766608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.766876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.767546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.771523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.788696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 854/1709] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:40:33.791830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.815539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a432-0631-4c96-8e42-3bdc8fdf6862 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.821966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 856/1710] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:33.827809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.832834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.840662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.840957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.843507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] PATH_INFO: `/v3/projects/788cd051d59e49489bf4c7df33d3ff00/users/f5162b36368241dd844934280fcc694e/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.854380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9d81b-835a-4b19-8fe7-d63c5844a078 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:33.870636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b432a6-ada6-4d7c-8f69-4e7dd90c66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:33.871459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b432a6-ada6-4d7c-8f69-4e7dd90c66c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.871749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b432a6-ada6-4d7c-8f69-4e7dd90c66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.872040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b432a6-ada6-4d7c-8f69-4e7dd90c66c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.872681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b432a6-ada6-4d7c-8f69-4e7dd90c66c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.874716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b432a6-ada6-4d7c-8f69-4e7dd90c66c2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.874872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=788cd051d59e49489bf4c7df33d3ff00, user_id=f5162b36368241dd844934280fcc694e, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:33.882275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:33.911035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 855/1711] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:33 2022] PUT /identity/v3/projects/788cd051d59e49489bf4c7df33d3ff00/users/f5162b36368241dd844934280fcc694e/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:33.914813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:33.917461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:33.937819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 857/1712] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:33 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:33.939342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:33.939632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:33.939909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:33.946257 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:34.036121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] MFA Rules not processed for user `f5162b36368241dd844934280fcc694e`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:34.037800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:34.043282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e32c7c-b8f6-478b-b1ab-a2c579391b0b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:34.186419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:34.186419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.186419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.186419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] PATH_INFO: `/v3/users/03ce5f7be7ff454a920a0771fc37fd9a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.190960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ce5f7be7ff454a920a0771fc37fd9a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:34.193841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:34.215714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 858/1713] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:34.216886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.265067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcda72c-ca2e-4578-bf00-36127cf5ff45 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:34.278398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ce5f7be7ff454a920a0771fc37fd9a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:34.280018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ce5f7be7ff454a920a0771fc37fd9a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:34.285700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:34.286421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.286706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.286981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.287636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:34.289848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:34.300172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ce5f7be7ff454a920a0771fc37fd9a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:34.311360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:34.332678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 859/1714] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:34 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:34.336724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.346931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:34.366427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ce5f7be7ff454a920a0771fc37fd9a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:34.390688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ff40-f78e-4380-81f5-83646d72147f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:34.399021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ce5f7be7ff454a920a0771fc37fd9a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:34.400886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2acb84-1e23-4e9a-94ed-725fe8dc0f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:34.402920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2acb84-1e23-4e9a-94ed-725fe8dc0f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.404310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2acb84-1e23-4e9a-94ed-725fe8dc0f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.405960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2acb84-1e23-4e9a-94ed-725fe8dc0f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.407888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2acb84-1e23-4e9a-94ed-725fe8dc0f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:34.415634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 856/1715] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:33 2022] DELETE /identity/v3/users/03ce5f7be7ff454a920a0771fc37fd9a => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:34.416556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.417204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2acb84-1e23-4e9a-94ed-725fe8dc0f85 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:34.418961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:34.477058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 860/1716] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:34 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:34.478956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.479252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.479530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.517363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:34.519210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:34.528071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:34.575136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442997f1-c854-4f48-9580-22cd23f35221 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:34.681836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 861/1717] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:34.687563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.721084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562155a5-9fcb-445b-a3d4-d249aece182e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:34.762806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:34.764968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.766536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.766822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.770012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:34.776416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:34.806837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:34.807487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.808408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.808408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.808709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:34.815504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:34.855600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 862/1718] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:34 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:34.860133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.870665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 857/1719] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:34.877994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:34.891300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.921781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939e0fc-8c65-46ce-b880-9010bfa3becc admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:34.933228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:34.935201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:34.935549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:34.935953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:34.938120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:34.941034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:34.969726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 858/1720] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:34.971472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:34.995738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a4ef6-9074-48e1-915e-9f2b551500b2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:35.007023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:35.009000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.009382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.010939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] PATH_INFO: `/v3/projects/963ff0d4c59748789dcc7caa0de88941/users/76d9f4e6de18429c96e7f66d71e452fb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.077803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963ff0d4c59748789dcc7caa0de88941, user_id=76d9f4e6de18429c96e7f66d71e452fb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:35.084699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:35.087194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff193d-8d25-4dfc-946a-c7c1a13bc0c2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:35.100752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cddc7d-2ac6-4c8b-b332-b89540e0da5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:35.101503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cddc7d-2ac6-4c8b-b332-b89540e0da5b service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.101815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cddc7d-2ac6-4c8b-b332-b89540e0da5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.101951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cddc7d-2ac6-4c8b-b332-b89540e0da5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.131494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 859/1721] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:34 2022] PUT /identity/v3/projects/963ff0d4c59748789dcc7caa0de88941/users/76d9f4e6de18429c96e7f66d71e452fb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:35.133971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:35.139686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cddc7d-2ac6-4c8b-b332-b89540e0da5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:35.142131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cddc7d-2ac6-4c8b-b332-b89540e0da5b service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:35.180917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 863/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:35.182366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.182660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.182941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.311737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:35.312660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:35.335400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2725c7e-9b64-4901-843b-d18d8120baa9 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:35.348753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:35.352338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.352571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.353898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.355952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:35.360596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:35.387698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 860/1723] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:35.388805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:35.396356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:35.422680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d44e6-0c9d-4498-9b46-83017ade3159 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:35.436093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:35.437172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.437463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.437742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.439009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:35.441301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:35.462518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 861/1724] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:35.462518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:35.489630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 864/1725] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:40:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:35.490797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:35.492491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04495462-55c3-47a6-8d30-09d66b195eff admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:35.503194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4cce8-801d-485c-8d23-f45310dc072b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:35.508032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c5de2-80b5-49c6-95a9-8461e00bde9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:35.508849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5de2-80b5-49c6-95a9-8461e00bde9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.509115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d7cd-75fd-41f3-9e77-09c83f7b1db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:35.509605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5de2-80b5-49c6-95a9-8461e00bde9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.509938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c5de2-80b5-49c6-95a9-8461e00bde9a admin admin] PATH_INFO: `/v3/projects/1b8d6ccf898d465ea985d3d3895a2db1/users/a17ab7c732ab48a49a793e1bf309efcb/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.510521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d7cd-75fd-41f3-9e77-09c83f7b1db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.511125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d7cd-75fd-41f3-9e77-09c83f7b1db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.511599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d7cd-75fd-41f3-9e77-09c83f7b1db2 admin admin] PATH_INFO: `/v3/projects/963ff0d4c59748789dcc7caa0de88941/users/76d9f4e6de18429c96e7f66d71e452fb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.514000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d7cd-75fd-41f3-9e77-09c83f7b1db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963ff0d4c59748789dcc7caa0de88941, user_id=76d9f4e6de18429c96e7f66d71e452fb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:35.517783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d7cd-75fd-41f3-9e77-09c83f7b1db2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:35.538489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 865/1726] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:35 2022] PUT /identity/v3/projects/963ff0d4c59748789dcc7caa0de88941/users/76d9f4e6de18429c96e7f66d71e452fb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:35.542040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.542239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.543870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.545111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c5de2-80b5-49c6-95a9-8461e00bde9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8d6ccf898d465ea985d3d3895a2db1, user_id=a17ab7c732ab48a49a793e1bf309efcb, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:35.551662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c5de2-80b5-49c6-95a9-8461e00bde9a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:35.583808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 862/1727] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:35 2022] PUT /identity/v3/projects/1b8d6ccf898d465ea985d3d3895a2db1/users/a17ab7c732ab48a49a793e1bf309efcb/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:35.585563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e090df-22c9-48d8-b2b6-0aab0d04b069 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.586689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e090df-22c9-48d8-b2b6-0aab0d04b069 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.587079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e090df-22c9-48d8-b2b6-0aab0d04b069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.626622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] MFA Rules not processed for user `43232b9b191047f9a3e6ebab7ad36f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:35.628056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:35.647754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:35.689169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e3e090df-22c9-48d8-b2b6-0aab0d04b069 None None] MFA Rules not processed for user `76d9f4e6de18429c96e7f66d71e452fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:35.690885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e3e090df-22c9-48d8-b2b6-0aab0d04b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:35.811166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 863/1728] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:35.812669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:35.812957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:35.813225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:35.839726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 866/1729] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:35.850111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:35.851943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:35.871302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] MFA Rules not processed for user `a17ab7c732ab48a49a793e1bf309efcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:35.873398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:35.988401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636d0e-c044-4c93-b272-366acbc21df0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:36.000846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:36.001584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:36.001868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:36.006088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:36.023973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 864/1730] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:36.025844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:36.027757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:36.065243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:36.132944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:36.135030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:36.144134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb021d-fe9d-46b2-8e12-98d5a11c6a36 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:36.156528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 867/1731] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:40:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:40:36.158843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:36.161650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:36.162185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:36.162382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:36.165332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:36.166287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:36.189565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:36.191163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:36.216698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 865/1732] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:36.217707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:36.368571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:36.486073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19f9f-5b95-4e4b-8e82-9134739d124e service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:36.490836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcffac-dbaf-4ac1-9177-ef694bafb3f6 service swift] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:36.507272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:36.507889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:36.508777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:36.508998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:36.509749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:36.518381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:36.518728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:36.519115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:36.519422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:36.519701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:36.551487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:36.553221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:36.587473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 868/1733] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:36.588458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:36.594873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:36.611336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 866/1734] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:36 2022] POST /identity/v3/projects => generated 379 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:36.615425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:36.625239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e36aa7-28a1-4956-9358-35e3f2bc4bd9 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:36.823094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:36.823830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:36.824131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:36.824388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] PATH_INFO: `/v3/projects/c6566aa21ff04589b57219c3be0baebb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:36.830089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6566aa21ff04589b57219c3be0baebb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:36.832563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:36.833089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f5cc9-d4a0-423f-997a-be62fe2716b8 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:36.843351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:36.844084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:36.844386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:36.844790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:36.852447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:37.029682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6566aa21ff04589b57219c3be0baebb', 'request_id': 'req-b12b68c8-f9d1-4cde-8f6f-21803e92f619'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:37.034633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:37.035690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:37.036860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:37.063105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6566aa21ff04589b57219c3be0baebb', 'request_id': 'req-b12b68c8-f9d1-4cde-8f6f-21803e92f619'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:37.075411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 867/1735] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:37.076294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:37.081519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6566aa21ff04589b57219c3be0baebb', 'request_id': 'req-b12b68c8-f9d1-4cde-8f6f-21803e92f619'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:37.115445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21614fd-9445-4a37-8cc1-ddb7a7d29d96 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:37.122784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] The token cache is being invalidate because project c6566aa21ff04589b57219c3be0baebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:37.123311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:37.125162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b12b68c8-f9d1-4cde-8f6f-21803e92f619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:37.127573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 869/1736] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:36 2022] DELETE /identity/v3/projects/c6566aa21ff04589b57219c3be0baebb => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:37.147140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2039e-fcdb-4de2-ab2f-5870ce5a09b4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.147808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2039e-fcdb-4de2-ab2f-5870ce5a09b4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.147808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2039e-fcdb-4de2-ab2f-5870ce5a09b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.258402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e5f2039e-fcdb-4de2-ab2f-5870ce5a09b4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:37.260187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e5f2039e-fcdb-4de2-ab2f-5870ce5a09b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:37.280256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 870/1737] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:37.281019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:37.281683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.281966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.286299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.286611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.286859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.295912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.297317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:37.299438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:37.348634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:37.366843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:37.382890 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 868/1738] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:37 2022] POST /identity/v3/users => generated 296 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:37.386945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:37.396096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:37.420512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:37.448740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 871/1739] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:37.452842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:37.475839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1c0c-d4fc-4c37-afd7-52f46ded6b6c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:37.494097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:37.496526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.496814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.497095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.500753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca948ff5-90a2-43de-8af2-15b35fb6f66e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:37.502215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:37.509343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:37.524277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:37.525025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.525309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.525690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.532610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:37.534367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:37.549391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:37.568486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 869/1740] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:37.573125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:37.618091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2df4e-5ed9-48e5-9fde-e243de463830 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:37.622937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 872/1741] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:40:37.630484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:37.638975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:37.642778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.643135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.643503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] PATH_INFO: `/v3/projects/f2f0015aa161493cb1ffa38823209d74/users/fe1b26457d5d4538881b0248a8d516c8/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.666755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fcdf7-76c3-4514-a677-478fe2e04ad2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:37.680922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:37.683041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:37.684166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:37.685735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:37.688292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:37.695522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:37.731833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f0015aa161493cb1ffa38823209d74, user_id=fe1b26457d5d4538881b0248a8d516c8, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:37.748666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:37.782105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 870/1742] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:37 2022] PUT /identity/v3/projects/f2f0015aa161493cb1ffa38823209d74/users/fe1b26457d5d4538881b0248a8d516c8/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:37.787508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:37.792168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 873/1743] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:37 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:37.794281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:38.087108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349912e9-b69c-40e8-9fc1-4ae18a1b64f0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:38.115283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:38.115283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.115283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.120383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.125996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:38.136113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:38.166191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83671875-16e3-4aa6-a9e5-104f0b3378aa admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:38.172031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:38.177171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.177171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.177171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.177171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:38.177171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:38.195507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 871/1744] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:40:38.197960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:38.253989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 874/1745] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:37 2022] POST /identity/v3/users => generated 287 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:38.258226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:38.263214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad948e5-a88f-41cc-bf3c-003ca035064c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:38.268925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96894-cbc3-4bb7-adfe-0e2d63c11de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:38.269820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96894-cbc3-4bb7-adfe-0e2d63c11de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.270111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96894-cbc3-4bb7-adfe-0e2d63c11de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.270464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96894-cbc3-4bb7-adfe-0e2d63c11de4 admin admin] PATH_INFO: `/v3/projects/f2f0015aa161493cb1ffa38823209d74/users/fe1b26457d5d4538881b0248a8d516c8/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.273770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e96894-cbc3-4bb7-adfe-0e2d63c11de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f0015aa161493cb1ffa38823209d74, user_id=fe1b26457d5d4538881b0248a8d516c8, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:38.290186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f0c31-c7ac-4684-985f-11debbc655c6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:38.300921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e96894-cbc3-4bb7-adfe-0e2d63c11de4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:38.305183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:38.306259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.308572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.309791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.314148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:38.316204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 872/1746] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:38 2022] PUT /identity/v3/projects/f2f0015aa161493cb1ffa38823209d74/users/fe1b26457d5d4538881b0248a8d516c8/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:38.319244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:38.334446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 875/1747] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:38.338331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:38.348059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.349202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.349487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.363161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099314b6-b28a-4877-a10e-e45a7091267e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:38.442968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] MFA Rules not processed for user `fe1b26457d5d4538881b0248a8d516c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:38.446447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:38.536645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:38.537311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.537592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.537868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.538722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:38.540445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:38.565209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 876/1748] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:38.571595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:38.602680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e1f86-c24d-4ef0-9018-1a1a41c3e81f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:38.614953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64583f-74d9-496b-9672-9b518b72fe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:38.615666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64583f-74d9-496b-9672-9b518b72fe79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.615947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64583f-74d9-496b-9672-9b518b72fe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.616221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64583f-74d9-496b-9672-9b518b72fe79 admin admin] PATH_INFO: `/v3/projects/b3ae25c245884edab1818d7077563373/users/aa16f4d30e3b49808d57a86842f82fb2/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.624435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:38.651183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64583f-74d9-496b-9672-9b518b72fe79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ae25c245884edab1818d7077563373, user_id=aa16f4d30e3b49808d57a86842f82fb2, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:38.659989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64583f-74d9-496b-9672-9b518b72fe79 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:38.700702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 877/1749] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:38 2022] PUT /identity/v3/projects/b3ae25c245884edab1818d7077563373/users/aa16f4d30e3b49808d57a86842f82fb2/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:38.715609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:38.715609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:38.716478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:38.734296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 873/1750] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:38.747653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:38.749400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:38.825878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] MFA Rules not processed for user `aa16f4d30e3b49808d57a86842f82fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:38.827233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:38.901089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:39.011044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ca09-5780-4f61-98d3-4b9ec114d170 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:39.061067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 878/1751] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:39.062222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:39.178311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:39.207155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:39.207446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:39.207725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:39.285860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ee73-b638-429a-ab6a-3b834375ba33 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:39.303475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:39.304128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:39.304405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:39.304676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:39.310676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:39.316109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:39.428828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 879/1752] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:39 2022] POST /identity/v3/projects => generated 373 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:39.433093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:39.471706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:39.475850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:39.514250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 874/1753] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:39.586400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:39.602267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8f79-fe74-4fd0-8a5b-7bd7c49551bf service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:39.629056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:39.643341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33ac9-73e7-4265-a2b4-cc7092ae1b03 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:39.661326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:39.664083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:39.664611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:39.665205 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:39.668378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:39.672442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:39.737745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:39.739952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:39.740293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:39.740571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:39.754365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:39.756602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 880/1754] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:39 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:39.757497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:39.786249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63170033-2686-4538-9413-54325d3463d7 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:39.812558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:39.818573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:39.818880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:39.819352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:39.821627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:39.826839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:39.882890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 881/1755] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:39.886335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:39.920695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6bdac-66f9-4263-a875-66f7244399b2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:39.940126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:39.940791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:39.942128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:39.942446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] PATH_INFO: `/v3/projects/2508048e92fc40118d801ccb8df2b39f/users/1094811437794569a0604e734aabcf6d/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:39.952205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:40.004982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:40.008387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:40.008678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2508048e92fc40118d801ccb8df2b39f, user_id=1094811437794569a0604e734aabcf6d, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:40.013822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:40.045393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 875/1756] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:40.080953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 882/1757] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:39 2022] PUT /identity/v3/projects/2508048e92fc40118d801ccb8df2b39f/users/1094811437794569a0604e734aabcf6d/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:40.082605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:40.100013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a42d8-a0f0-464b-84e2-356618cdb468 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:40.296297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53af13e-e4fd-4112-a3b3-1f47ef2a47dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:40.297149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53af13e-e4fd-4112-a3b3-1f47ef2a47dc admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:40.297771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53af13e-e4fd-4112-a3b3-1f47ef2a47dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:40.298362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53af13e-e4fd-4112-a3b3-1f47ef2a47dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:40.299755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53af13e-e4fd-4112-a3b3-1f47ef2a47dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:40.306414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53af13e-e4fd-4112-a3b3-1f47ef2a47dc admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:40.342488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 876/1758] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:40.343669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:40.371075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557d509-cf4e-4c65-9d0e-688c8643c378 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:40.384448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:40.385146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:40.385343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:40.385612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] PATH_INFO: `/v3/projects/2508048e92fc40118d801ccb8df2b39f/users/1094811437794569a0604e734aabcf6d/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:40.390196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2508048e92fc40118d801ccb8df2b39f, user_id=1094811437794569a0604e734aabcf6d, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:40.397510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:40.424715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 883/1759] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:40 2022] PUT /identity/v3/projects/2508048e92fc40118d801ccb8df2b39f/users/1094811437794569a0604e734aabcf6d/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:40.436096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557188d-60aa-4a2b-81d8-5d9e17349a60 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:40.436432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557188d-60aa-4a2b-81d8-5d9e17349a60 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:40.436722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557188d-60aa-4a2b-81d8-5d9e17349a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:40.497512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0557188d-60aa-4a2b-81d8-5d9e17349a60 None None] MFA Rules not processed for user `1094811437794569a0604e734aabcf6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:40.498793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0557188d-60aa-4a2b-81d8-5d9e17349a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:40.704526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 877/1760] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:40.712108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:40.714016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:40.920003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609668e7-73f5-4f0b-ad9b-6917c259d884 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:40.933246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:40.933961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:40.934248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:40.934564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:40.962504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:40.964004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:40.982194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 884/1761] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:41.557334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:41.558429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:41.558984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:41.615304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:41.622095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:41.794658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 878/1762] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Aug 9 02:40:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:41.874208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:41.875698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:41.910876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4d90d-793f-4931-b066-0a8f8f2b06e0 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:41.935850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8ae66-40a2-4459-88dd-6574c4d4b810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:41.958683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8ae66-40a2-4459-88dd-6574c4d4b810 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:41.960308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8ae66-40a2-4459-88dd-6574c4d4b810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:41.964900 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8ae66-40a2-4459-88dd-6574c4d4b810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:42.022251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8ae66-40a2-4459-88dd-6574c4d4b810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:42.024313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8ae66-40a2-4459-88dd-6574c4d4b810 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:42.069884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 885/1763] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:42.087755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:42.149326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:42.311146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e8223-648e-4d00-beab-259519c8fcb9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:42.330016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:42.330720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:42.342188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:42.342188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] PATH_INFO: `/v3/users/aa16f4d30e3b49808d57a86842f82fb2` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:42.350677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa16f4d30e3b49808d57a86842f82fb2)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:42.359087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:42.387897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b396c32-ba27-4d03-8009-d35fced2b50b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:42.388243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b396c32-ba27-4d03-8009-d35fced2b50b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:42.388581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b396c32-ba27-4d03-8009-d35fced2b50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:42.470590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-9b396c32-ba27-4d03-8009-d35fced2b50b None None] MFA Rules not processed for user `43232b9b191047f9a3e6ebab7ad36f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:42.473720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9b396c32-ba27-4d03-8009-d35fced2b50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:42.531857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa16f4d30e3b49808d57a86842f82fb2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:42.539741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa16f4d30e3b49808d57a86842f82fb2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:42.559310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa16f4d30e3b49808d57a86842f82fb2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:42.584853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa16f4d30e3b49808d57a86842f82fb2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:42.640823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa16f4d30e3b49808d57a86842f82fb2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:42.665963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 886/1764] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:40:42.692028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 879/1765] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:42 2022] DELETE /identity/v3/users/aa16f4d30e3b49808d57a86842f82fb2 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:42.693349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:42.693645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:42.694864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:42.827085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:42.828347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:42.964564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:43.110708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 887/1766] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:40:43.137514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:43.139062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:43.308646 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73921234-14fe-420c-937d-a5b501ce6c4b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:43.331038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d08c14-f59a-4769-bc47-182661ee13c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:43.331747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d08c14-f59a-4769-bc47-182661ee13c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:43.332031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d08c14-f59a-4769-bc47-182661ee13c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:43.332461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d08c14-f59a-4769-bc47-182661ee13c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:43.351378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d08c14-f59a-4769-bc47-182661ee13c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:43.356638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d08c14-f59a-4769-bc47-182661ee13c5 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:43.387255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 880/1767] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:43.880025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:43.881762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:43.909040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca94a3-bc14-49a7-9a27-4215fba6be81 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:43.932689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:43.938676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:43.942664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:43.943077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] PATH_INFO: `/v3/users/70c579b22f7b4160bec240b976b64526` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:43.953368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c579b22f7b4160bec240b976b64526)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:43.955514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:44.100788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c579b22f7b4160bec240b976b64526'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:44.118362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c579b22f7b4160bec240b976b64526'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:44.119657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c579b22f7b4160bec240b976b64526'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:44.167597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c579b22f7b4160bec240b976b64526'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:44.233039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223dc6e-e25d-4064-bb3e-d26adf3f11ef None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:44.233722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223dc6e-e25d-4064-bb3e-d26adf3f11ef None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:44.233722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223dc6e-e25d-4064-bb3e-d26adf3f11ef None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:44.235299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 881/1768] 10.222.0.20 () {54 vars in 912 bytes} [Tue Aug 9 02:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:44.236190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a6ce3b33-9889-4ccc-81c4-9bdb01d353b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c579b22f7b4160bec240b976b64526'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:44.243718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:44.244005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:44.244280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:44.267682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 888/1769] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:43 2022] DELETE /identity/v3/users/70c579b22f7b4160bec240b976b64526 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:44.273087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:44.273772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:44.274232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:44.326798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:44.328334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:44.335351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:44.336637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:44.533660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:44.567105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 889/1770] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:40:44.587252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 882/1771] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:40:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 09 02:40:44.608332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:44.610417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:44.629984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:44.640484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:44.680407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cf52c-141a-4c07-92a7-c5afad5ace26 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:44.697211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:44.698281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:44.699485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399affe4-b751-472e-a73b-e76557ffe4d3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:44.700020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:44.701088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:44.720595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:44.721504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:44.722023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:44.722444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:44.727577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:44.730259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:44.743956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:44.746980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:44.753283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 890/1772] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:44.793786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 883/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:45.390112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:45.394136 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:45.442124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4d67-d421-4f1b-ae67-e804a7c31bbe service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:45.454335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:45.456602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:45.459150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:45.460857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] PATH_INFO: `/v3/users/f5162b36368241dd844934280fcc694e` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:45.465227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5162b36368241dd844934280fcc694e)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:45.472316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:45.565537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:45.614562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5162b36368241dd844934280fcc694e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:45.638395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5162b36368241dd844934280fcc694e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:45.639460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5162b36368241dd844934280fcc694e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:45.651064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5162b36368241dd844934280fcc694e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:45.687923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d7329794-d5f5-4175-a5b6-f99af8b6c74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5162b36368241dd844934280fcc694e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:45.695368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:45.697368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:45.725981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 891/1774] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:45 2022] DELETE /identity/v3/users/f5162b36368241dd844934280fcc694e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:45.734605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45409b-32d8-4dfa-8480-7f6ee4c8cc03 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:45.734958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:45.738109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:45.739952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:45.866132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:45.867754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:46.015499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:46.018017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:46.022520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:46.022942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] PATH_INFO: `/v3/projects/b3ae25c245884edab1818d7077563373` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:46.025876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ae25c245884edab1818d7077563373)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:46.028690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:46.107352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 892/1775] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:46.122216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:46.132114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:46.215798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ae25c245884edab1818d7077563373', 'request_id': 'req-7936456a-7c68-4a8e-af22-9d41e427556f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.233328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ae25c245884edab1818d7077563373', 'request_id': 'req-7936456a-7c68-4a8e-af22-9d41e427556f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.254935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ae25c245884edab1818d7077563373', 'request_id': 'req-7936456a-7c68-4a8e-af22-9d41e427556f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.285262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] The token cache is being invalidate because project b3ae25c245884edab1818d7077563373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:46.287169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.289091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.293379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 884/1776] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:45 2022] DELETE /identity/v3/projects/b3ae25c245884edab1818d7077563373 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:40:46.296864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:46.298547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:46.341958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5e2de-6c79-4cc4-9b7e-01d8cac53a9c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:46.620776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936456a-7c68-4a8e-af22-9d41e427556f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:46.634281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:46.634964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:46.635250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:46.635531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] PATH_INFO: `/v3/users/1094811437794569a0604e734aabcf6d` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:46.643154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1094811437794569a0604e734aabcf6d)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:46.645569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:46.646569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:46.647280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:46.647765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:46.648186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:46.652164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:46.662334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:46.702444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:46.705748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:46.734308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 893/1777] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:46.794059 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:46.795972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:46.805750 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:46.810290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1094811437794569a0604e734aabcf6d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.811312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c767d10-4986-4f21-b76a-783e772498e5 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:46.814388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1094811437794569a0604e734aabcf6d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.817943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:46.824518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:46.825810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:46.826313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] PATH_INFO: `/v3/users/692335d9e9e64c0f8e1c581ed6048fa9` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:46.828084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=692335d9e9e64c0f8e1c581ed6048fa9)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:46.830569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:46.842194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1094811437794569a0604e734aabcf6d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.869002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1094811437794569a0604e734aabcf6d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.930697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9bc4875d-0627-4ddc-8599-a7641904838c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1094811437794569a0604e734aabcf6d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.961713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 885/1778] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:46 2022] DELETE /identity/v3/users/1094811437794569a0604e734aabcf6d => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:46.969622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '692335d9e9e64c0f8e1c581ed6048fa9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.997404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '692335d9e9e64c0f8e1c581ed6048fa9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:46.998536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '692335d9e9e64c0f8e1c581ed6048fa9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:47.065233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '692335d9e9e64c0f8e1c581ed6048fa9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:47.144155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '692335d9e9e64c0f8e1c581ed6048fa9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:47.156282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 894/1779] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:46 2022] DELETE /identity/v3/users/692335d9e9e64c0f8e1c581ed6048fa9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:47.161190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790436e-c9d2-4fb3-813b-f53f07b1c658 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:47.161538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790436e-c9d2-4fb3-813b-f53f07b1c658 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:47.161747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790436e-c9d2-4fb3-813b-f53f07b1c658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:47.212058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c790436e-c9d2-4fb3-813b-f53f07b1c658 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:47.214858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c790436e-c9d2-4fb3-813b-f53f07b1c658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:47.402745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 886/1780] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:47.583961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:47.586516 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:47.740346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87410d25-af8f-41c4-9251-0f13b8a4e797 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:47.749278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a57d6-57e0-48c6-bf40-790c4c624aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:47.751084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a57d6-57e0-48c6-bf40-790c4c624aab service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:47.751627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a57d6-57e0-48c6-bf40-790c4c624aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:47.752009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a57d6-57e0-48c6-bf40-790c4c624aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:47.768515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a57d6-57e0-48c6-bf40-790c4c624aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:47.771661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a57d6-57e0-48c6-bf40-790c4c624aab service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:47.793251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 895/1781] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:48.312025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e1d74-3eda-440c-bc84-afd912e094a5 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:48.313241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e1d74-3eda-440c-bc84-afd912e094a5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:48.313746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e1d74-3eda-440c-bc84-afd912e094a5 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:48.316895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 887/1782] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:48.519349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:48.519614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:48.519810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:48.592905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:48.597218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:48.626183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e1d74-3eda-440c-bc84-afd912e094a5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:48.628096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-100e1d74-3eda-440c-bc84-afd912e094a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:48.669090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e1d74-3eda-440c-bc84-afd912e094a5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:48.695581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:48.696004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:48.696283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:48.696550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] PATH_INFO: `/v3/projects/0dc5596725f34e7781ec7b1ae4c7fbb8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:48.698005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc5596725f34e7781ec7b1ae4c7fbb8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:48.700036 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:48.845957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 896/1783] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:40:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:48.847922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:48.849710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:48.855382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc5596725f34e7781ec7b1ae4c7fbb8', 'request_id': 'req-fdf4bd1f-2902-489d-a947-6cc24c6037ab'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:48.867234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc5596725f34e7781ec7b1ae4c7fbb8', 'request_id': 'req-fdf4bd1f-2902-489d-a947-6cc24c6037ab'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:48.885303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc5596725f34e7781ec7b1ae4c7fbb8', 'request_id': 'req-fdf4bd1f-2902-489d-a947-6cc24c6037ab'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:48.891194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf253692-db0b-4cb7-a38b-cfcf7a58c940 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:48.914095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] The token cache is being invalidate because project 0dc5596725f34e7781ec7b1ae4c7fbb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:48.915447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:48.916699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:48.917920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:48.921696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 888/1784] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:48 2022] DELETE /identity/v3/projects/0dc5596725f34e7781ec7b1ae4c7fbb8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:48.927184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:48.928520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:48.968256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:49.012505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:49.013186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:49.013472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:49.013767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] PATH_INFO: `/v3/projects/788cd051d59e49489bf4c7df33d3ff00` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:49.015143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=788cd051d59e49489bf4c7df33d3ff00)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:49.017309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:49.097270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788cd051d59e49489bf4c7df33d3ff00', 'request_id': 'req-ef9ea28e-5fb3-4239-98ff-7584f9258637'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.111278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4bd1f-2902-489d-a947-6cc24c6037ab admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:49.117803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788cd051d59e49489bf4c7df33d3ff00', 'request_id': 'req-ef9ea28e-5fb3-4239-98ff-7584f9258637'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.146269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788cd051d59e49489bf4c7df33d3ff00', 'request_id': 'req-ef9ea28e-5fb3-4239-98ff-7584f9258637'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.185699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] The token cache is being invalidate because project 788cd051d59e49489bf4c7df33d3ff00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:49.202021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.202021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.202021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 897/1785] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:48 2022] DELETE /identity/v3/projects/788cd051d59e49489bf4c7df33d3ff00 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:49.205376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:49.208821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:49.246824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:49.249349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:49.249349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:49.249349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] PATH_INFO: `/v3/users/76d9f4e6de18429c96e7f66d71e452fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:49.250118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d9f4e6de18429c96e7f66d71e452fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:49.261580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:49.360085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d9f4e6de18429c96e7f66d71e452fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.363615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d9f4e6de18429c96e7f66d71e452fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.366448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ea28e-5fb3-4239-98ff-7584f9258637 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:49.379691 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:49.380403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d9f4e6de18429c96e7f66d71e452fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.380535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:49.380752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:49.380952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] PATH_INFO: `/v3/users/a17ab7c732ab48a49a793e1bf309efcb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:49.382410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17ab7c732ab48a49a793e1bf309efcb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:49.384802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:49.404441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d9f4e6de18429c96e7f66d71e452fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.440200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d9f4e6de18429c96e7f66d71e452fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.455532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 889/1786] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:48 2022] DELETE /identity/v3/users/76d9f4e6de18429c96e7f66d71e452fb => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:49.459981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:49.488632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:49.521292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:49.531237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17ab7c732ab48a49a793e1bf309efcb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.549004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17ab7c732ab48a49a793e1bf309efcb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.549831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17ab7c732ab48a49a793e1bf309efcb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.580425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17ab7c732ab48a49a793e1bf309efcb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.639456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:49.641731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17ab7c732ab48a49a793e1bf309efcb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:49.663037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 898/1787] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:49 2022] DELETE /identity/v3/users/a17ab7c732ab48a49a793e1bf309efcb => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:49.664607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:49.668321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:49.758722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd237-a34a-43b1-8271-77db63c1e789 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:49.889567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6265d7-1a31-4aed-9677-a8ce8ab732e5 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:49.907517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:49.909438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:49.910132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:49.910132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] PATH_INFO: `/v3/projects/2508048e92fc40118d801ccb8df2b39f` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:49.915527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2508048e92fc40118d801ccb8df2b39f)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:49.919887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:50.003129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:50.003868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:50.007898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:50.008039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:50.029833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:50.067159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:50.070805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65263de9-4346-4914-9faf-247e09af80fb service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:50.094099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2508048e92fc40118d801ccb8df2b39f', 'request_id': 'req-04eb5fd9-2ffc-4008-87ca-23905efa3688'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:50.100692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 890/1788] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:50.118948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2508048e92fc40118d801ccb8df2b39f', 'request_id': 'req-04eb5fd9-2ffc-4008-87ca-23905efa3688'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:50.177037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2508048e92fc40118d801ccb8df2b39f', 'request_id': 'req-04eb5fd9-2ffc-4008-87ca-23905efa3688'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:50.206772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] The token cache is being invalidate because project 2508048e92fc40118d801ccb8df2b39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:50.207746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:50.210341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-04eb5fd9-2ffc-4008-87ca-23905efa3688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:50.213969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 899/1789] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:49 2022] DELETE /identity/v3/projects/2508048e92fc40118d801ccb8df2b39f => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:50.221280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f928c1-df2c-4d5c-b095-aef6fffd3c87 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:50.221959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f928c1-df2c-4d5c-b095-aef6fffd3c87 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:50.222379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f928c1-df2c-4d5c-b095-aef6fffd3c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:50.275052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b4f928c1-df2c-4d5c-b095-aef6fffd3c87 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:50.277178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b4f928c1-df2c-4d5c-b095-aef6fffd3c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:50.478065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859e237-be94-434f-874e-a8eae30f254b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:50.479429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859e237-be94-434f-874e-a8eae30f254b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:50.479429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859e237-be94-434f-874e-a8eae30f254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:50.558764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8859e237-be94-434f-874e-a8eae30f254b None None] MFA Rules not processed for user `2aaa8a7f386e4ccc9ab5c356cdf1704f`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:50.561124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8859e237-be94-434f-874e-a8eae30f254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:50.600969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 891/1790] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:50.608880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:50.609475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:50.610051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:50.660299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:50.678947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:50.681011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:50.696268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 892/1791] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:40:50.701168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:50.749340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae3963-314b-4084-9a56-5de454914d44 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:50.763877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 900/1792] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:50.764841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:50.765455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:50.765734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:50.766007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:50.780676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859e237-be94-434f-874e-a8eae30f254b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:50.782712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8859e237-be94-434f-874e-a8eae30f254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:50.784619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:50.785028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:50.787098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:50.833679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 893/1793] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:40:50.837783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:50.841053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:50.875748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969a77b-fbfe-439c-a078-8056d6e1c1e5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:50.886565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:50.887394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:50.887817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:50.890050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] PATH_INFO: `/v3/projects/16722152c8004b609a4d900ba50b3085` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:50.893134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16722152c8004b609a4d900ba50b3085)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:50.895133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:51.024013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859e237-be94-434f-874e-a8eae30f254b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:51.044690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:51.045934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:51.048734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:51.049412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:51.061771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:51.087167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16722152c8004b609a4d900ba50b3085', 'request_id': 'req-42160ce5-63c5-4679-84b9-1944ef57f27c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.096748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:51.103764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:51.118767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16722152c8004b609a4d900ba50b3085', 'request_id': 'req-42160ce5-63c5-4679-84b9-1944ef57f27c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.136598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16722152c8004b609a4d900ba50b3085', 'request_id': 'req-42160ce5-63c5-4679-84b9-1944ef57f27c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.155300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 901/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:51.156862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:51.170513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] The token cache is being invalidate because project 16722152c8004b609a4d900ba50b3085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:51.170825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.172217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.173807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 894/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:50 2022] DELETE /identity/v3/projects/16722152c8004b609a4d900ba50b3085 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:51.177089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:51.178709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:51.218589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a14d1-9c28-49ad-ab56-87d28634fbe6 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:51.324677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42160ce5-63c5-4679-84b9-1944ef57f27c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:51.336759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:51.337489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:51.337790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:51.338112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] PATH_INFO: `/v3/projects/963ff0d4c59748789dcc7caa0de88941` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:51.340561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] RBAC: Authorizing `identity:delete_project(project_id=963ff0d4c59748789dcc7caa0de88941)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:51.351792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:51.423031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:51.423677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:51.423956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:51.424226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:51.428895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:51.435422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:51.464296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963ff0d4c59748789dcc7caa0de88941', 'request_id': 'req-e29b3db5-5e29-47c0-bd42-a54a9a99479b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.476751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963ff0d4c59748789dcc7caa0de88941', 'request_id': 'req-e29b3db5-5e29-47c0-bd42-a54a9a99479b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.492217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963ff0d4c59748789dcc7caa0de88941', 'request_id': 'req-e29b3db5-5e29-47c0-bd42-a54a9a99479b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.512813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] The token cache is being invalidate because project 963ff0d4c59748789dcc7caa0de88941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:51.513109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.514823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.520498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 902/1797] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:51 2022] POST /identity/v3/projects => generated 377 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:51.521487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:51.523178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 895/1797] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:51 2022] DELETE /identity/v3/projects/963ff0d4c59748789dcc7caa0de88941 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:51.524139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:51.526127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:51.529765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:51.618135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:51.630107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:51.639032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3db5-5e29-47c0-bd42-a54a9a99479b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:51.647276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:51.648461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:51.648839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:51.649218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] PATH_INFO: `/v3/projects/1b8d6ccf898d465ea985d3d3895a2db1` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:51.650636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8d6ccf898d465ea985d3d3895a2db1)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:51.653522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:51.665280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95743fb6-b87d-497f-bd8d-0d21a6401954 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:51.682066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:51.682762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:51.683042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:51.683318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] PATH_INFO: `/v3/users/a708a93803b346acb235375988584c93` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:51.684501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a708a93803b346acb235375988584c93)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:51.686522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:51.755535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8d6ccf898d465ea985d3d3895a2db1', 'request_id': 'req-7636c95a-bb57-4d1f-9073-75f4726fe2c6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.768150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8d6ccf898d465ea985d3d3895a2db1', 'request_id': 'req-7636c95a-bb57-4d1f-9073-75f4726fe2c6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.777015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a708a93803b346acb235375988584c93'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.783845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8d6ccf898d465ea985d3d3895a2db1', 'request_id': 'req-7636c95a-bb57-4d1f-9073-75f4726fe2c6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.797975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a708a93803b346acb235375988584c93'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.800414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a708a93803b346acb235375988584c93'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.821542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] The token cache is being invalidate because project 1b8d6ccf898d465ea985d3d3895a2db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:51.822195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.824809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.826222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a708a93803b346acb235375988584c93'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.831314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 896/1798] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:51 2022] DELETE /identity/v3/projects/1b8d6ccf898d465ea985d3d3895a2db1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:51.836345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:51.840155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:51.860927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a9a148d3-bcb7-4c13-8c27-77541e923a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a708a93803b346acb235375988584c93'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:51.873963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 903/1799] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:51 2022] DELETE /identity/v3/users/a708a93803b346acb235375988584c93 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:51.881046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:51.884774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:51.885295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:51.939467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:51.940888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:51.997406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c95a-bb57-4d1f-9073-75f4726fe2c6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.077258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 904/1800] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:52.079272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.081184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.107618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2afb-bb36-477b-8256-71ba18b86fa4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.118511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911bfb6-787d-4d36-b2f6-27587dbdb36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.119332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911bfb6-787d-4d36-b2f6-27587dbdb36c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.120149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911bfb6-787d-4d36-b2f6-27587dbdb36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.120521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911bfb6-787d-4d36-b2f6-27587dbdb36c admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.121263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911bfb6-787d-4d36-b2f6-27587dbdb36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.124823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911bfb6-787d-4d36-b2f6-27587dbdb36c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.126646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.127553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.127553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.127840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] PATH_INFO: `/v3/users/fe1b26457d5d4538881b0248a8d516c8` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.129071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1b26457d5d4538881b0248a8d516c8)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.131019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.166445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 897/1801] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:51 2022] POST /identity/v3/users => generated 289 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:52.170681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc0fe8-34bd-4c28-9b5a-5535575a2476 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.171159 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc0fe8-34bd-4c28-9b5a-5535575a2476 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.171531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc0fe8-34bd-4c28-9b5a-5535575a2476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.200460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-ebbc0fe8-34bd-4c28-9b5a-5535575a2476 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:52.202307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ebbc0fe8-34bd-4c28-9b5a-5535575a2476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.212598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 898/1802] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:52.214958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.215692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.215799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.240212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1b26457d5d4538881b0248a8d516c8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:52.250652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:52.254179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] MFA Rules not processed for user `192a8abd4c7a4535a42bea7648535dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:52.254879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1b26457d5d4538881b0248a8d516c8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:52.255481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.255941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1b26457d5d4538881b0248a8d516c8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:52.267575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1b26457d5d4538881b0248a8d516c8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:52.271129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:52.293599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1b26457d5d4538881b0248a8d516c8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:52.307352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 905/1803] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:52 2022] DELETE /identity/v3/users/fe1b26457d5d4538881b0248a8d516c8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:52.308514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.314958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.364862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 899/1804] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:52.366313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.367575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.409089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bf731-ac9f-46a2-a9d8-824ae618bba4 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.472532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564ce4-7346-4245-82af-9b48aa149666 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.482720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5100-54e7-4d4f-a94e-2b8f8f99f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.484148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5100-54e7-4d4f-a94e-2b8f8f99f033 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.484645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5100-54e7-4d4f-a94e-2b8f8f99f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.485188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5100-54e7-4d4f-a94e-2b8f8f99f033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.486391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a5100-54e7-4d4f-a94e-2b8f8f99f033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.489936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a5100-54e7-4d4f-a94e-2b8f8f99f033 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.505694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 906/1805] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:52.507655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.508839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.509470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.532944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.533668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.533901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.534141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.534943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:52.535244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.536870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.557942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] MFA Rules not processed for user `192a8abd4c7a4535a42bea7648535dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:52.560086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.572373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 900/1806] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:52.572373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.587023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a2e46-69d4-4e09-8c8e-770c3315810e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.602661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.603786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.604066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.604335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.605583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.607237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.651368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 901/1807] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:52.652338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.663906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bffb24-2d44-4f3d-9127-e936d2f75b29 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.669668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.670295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.670575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.670848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.671470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.673774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.709700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 907/1808] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:52.709700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.715542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 902/1809] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:52 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:52.716435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.733614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd33da2-eadc-4233-9ef1-8906ace470f0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.835679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.836662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.836946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.837221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] PATH_INFO: `/v3/projects/d777960bc55f4471abd193a7df4a6200/users/0e155fba6fe445cfacc865480348c3ae/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.851814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66bd4b-4f65-44ae-ac84-2bfc0fe990aa admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:52.857923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d777960bc55f4471abd193a7df4a6200, user_id=0e155fba6fe445cfacc865480348c3ae, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.860582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88b581-0033-4876-92e6-4cc0fa05bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:52.860810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.861217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b581-0033-4876-92e6-4cc0fa05bcef admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.861502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b581-0033-4876-92e6-4cc0fa05bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.861777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88b581-0033-4876-92e6-4cc0fa05bcef admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:52.863288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88b581-0033-4876-92e6-4cc0fa05bcef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:52.865867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88b581-0033-4876-92e6-4cc0fa05bcef admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:52.884745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:52.897460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 908/1810] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:52 2022] PUT /identity/v3/projects/d777960bc55f4471abd193a7df4a6200/users/0e155fba6fe445cfacc865480348c3ae/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:52.900866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:52.903342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:52.915707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:52.920867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 903/1811] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:52 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:52.924583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574011-6596-48bd-96c6-c976870b31af None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:52.924877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574011-6596-48bd-96c6-c976870b31af None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:52.926319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574011-6596-48bd-96c6-c976870b31af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.003457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-42574011-6596-48bd-96c6-c976870b31af None None] MFA Rules not processed for user `0e155fba6fe445cfacc865480348c3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:53.004193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-42574011-6596-48bd-96c6-c976870b31af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.012293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7e0e2-ff34-4f75-9898-ffa7c01878a3 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.027771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.028426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.028948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.029220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.153319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:53.159152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:53.182462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 909/1812] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:53.184338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:53.196126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 904/1813] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:53.197215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574011-6596-48bd-96c6-c976870b31af None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:53.201436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-42574011-6596-48bd-96c6-c976870b31af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.352602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb74b8-94ae-44b4-9ccd-696efc6022ec service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.352832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574011-6596-48bd-96c6-c976870b31af None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.365441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.366306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.366562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.366686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.367416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.368132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.368418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] PATH_INFO: `/v3/projects/da837c9180b942c4a723ae9c24dc6fb7` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.369372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.369965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:53.371683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:53.371923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da837c9180b942c4a723ae9c24dc6fb7)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:53.377217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:53.405912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 910/1814] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:53.407860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:53.410115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.440471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeff8df-5c2c-4120-b944-c30f91be4cfa admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.456121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.457012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.457199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.457509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] PATH_INFO: `/v3/projects/f2f0015aa161493cb1ffa38823209d74` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.459264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f0015aa161493cb1ffa38823209d74)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:53.461584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:53.492878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da837c9180b942c4a723ae9c24dc6fb7', 'request_id': 'req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.521276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da837c9180b942c4a723ae9c24dc6fb7', 'request_id': 'req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.532311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da837c9180b942c4a723ae9c24dc6fb7', 'request_id': 'req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.576526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:53.584102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] The token cache is being invalidate because project da837c9180b942c4a723ae9c24dc6fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:53.584451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.585351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.588034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 905/1815] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:53 2022] DELETE /identity/v3/projects/da837c9180b942c4a723ae9c24dc6fb7 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:53.588954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:53.591249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.597139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f0015aa161493cb1ffa38823209d74', 'request_id': 'req-9bc52985-e084-471f-90c8-971e7ef86663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.618606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f0015aa161493cb1ffa38823209d74', 'request_id': 'req-9bc52985-e084-471f-90c8-971e7ef86663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.620052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:53.628318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f0015aa161493cb1ffa38823209d74', 'request_id': 'req-9bc52985-e084-471f-90c8-971e7ef86663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.644695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] The token cache is being invalidate because project f2f0015aa161493cb1ffa38823209d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:40:53.645122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.645887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:53.652448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 911/1816] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:40:53 2022] DELETE /identity/v3/projects/f2f0015aa161493cb1ffa38823209d74 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:53.653439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:53.654901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.731809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf3d3a-982e-4e39-8c28-45dcdc131dc0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.745559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab57ef-ee20-4e72-9f58-46817a81b667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.746585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab57ef-ee20-4e72-9f58-46817a81b667 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.746857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab57ef-ee20-4e72-9f58-46817a81b667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.747123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab57ef-ee20-4e72-9f58-46817a81b667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.747767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab57ef-ee20-4e72-9f58-46817a81b667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:53.752593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab57ef-ee20-4e72-9f58-46817a81b667 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:53.760177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc52985-e084-471f-90c8-971e7ef86663 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.774082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73628890-fa65-4714-8b95-ff03328fd260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.774747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73628890-fa65-4714-8b95-ff03328fd260 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.775023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73628890-fa65-4714-8b95-ff03328fd260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.775291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73628890-fa65-4714-8b95-ff03328fd260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.775952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73628890-fa65-4714-8b95-ff03328fd260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:53.777574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73628890-fa65-4714-8b95-ff03328fd260 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:53.794231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 912/1817] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:53.797424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.797927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.798440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.829038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 906/1818] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:53 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:53.830738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.830738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.831027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:53.832020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:53.833610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.865571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:53.866856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:53.962553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 907/1819] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:53.963491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:53.989831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d9a48-8ac7-47ad-8974-57e322ce73d9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:53.995365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f512a-f80a-4973-a7f9-7fc5dcac869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:53.996518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f512a-f80a-4973-a7f9-7fc5dcac869b admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:53.996850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f512a-f80a-4973-a7f9-7fc5dcac869b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:53.997124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f512a-f80a-4973-a7f9-7fc5dcac869b admin admin] PATH_INFO: `/v3/projects/39ce31bd95aa4be1ae98f616475afe83/users/ce841a25351d4679a03205fbb66d5f93/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.004860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 913/1820] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:54.004860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.004860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:54.018618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16480fe-d942-45e4-8b50-82389bae1954 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.023959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f512a-f80a-4973-a7f9-7fc5dcac869b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ce31bd95aa4be1ae98f616475afe83, user_id=ce841a25351d4679a03205fbb66d5f93, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.031394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f512a-f80a-4973-a7f9-7fc5dcac869b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.035297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a98b7d-6fc1-4533-9abd-d7e77bf9f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.035957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a98b7d-6fc1-4533-9abd-d7e77bf9f173 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.036642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a98b7d-6fc1-4533-9abd-d7e77bf9f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.036929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a98b7d-6fc1-4533-9abd-d7e77bf9f173 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.037723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a98b7d-6fc1-4533-9abd-d7e77bf9f173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.043140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a98b7d-6fc1-4533-9abd-d7e77bf9f173 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.069951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 908/1821] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:53 2022] PUT /identity/v3/projects/39ce31bd95aa4be1ae98f616475afe83/users/ce841a25351d4679a03205fbb66d5f93/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:54.069951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.070362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.070362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.081194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 914/1822] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:53 2022] POST /identity/v3/users => generated 295 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:54.082712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b06b66-b3c2-4e32-9d80-35182fb97fed None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.083017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b06b66-b3c2-4e32-9d80-35182fb97fed None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.083354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b06b66-b3c2-4e32-9d80-35182fb97fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.112495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-70b06b66-b3c2-4e32-9d80-35182fb97fed None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:54.114391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-70b06b66-b3c2-4e32-9d80-35182fb97fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:54.130852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:54.132094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:54.189090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:54.250230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 915/1823] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:54.251587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.251825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.252093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.285722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 909/1824] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:54.285722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.305839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2aabd-31e1-4524-b230-b83ae8cce317 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.306373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] MFA Rules not processed for user `ce841a25351d4679a03205fbb66d5f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:54.307673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:54.314839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.315475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.315800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.316226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.316933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.320641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.356374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 910/1825] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:54.357443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.368736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:54.379572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8fe62-d67f-4b09-99c7-89ce1ec31442 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.401944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.409696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.409696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.409696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.409696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:54.409696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.410146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.446103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 911/1826] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:54.447765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.468472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc59ad-b005-478f-b5f3-209516365055 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.480018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 916/1827] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:54.480844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.499495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.500459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.500459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.500709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.501393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:40:54.501701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.503632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.535706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea7cf1-78b2-451e-a6b9-9755da332606 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.538699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 912/1828] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:40:54.539659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.554913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.555614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.555892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.556162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] PATH_INFO: `/v3/projects/58b6ff2e844e4041a64946b89fe5ff29/users/006e8964a50a417d8d5cda9d8bc422a1/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.562736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d4a0f-90a0-4c37-852a-5f024e3c91d2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.573102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.573738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.574013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.574351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.574976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.576881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.592440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b6ff2e844e4041a64946b89fe5ff29, user_id=006e8964a50a417d8d5cda9d8bc422a1, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.596097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.635323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 917/1829] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:54 2022] PUT /identity/v3/projects/58b6ff2e844e4041a64946b89fe5ff29/users/006e8964a50a417d8d5cda9d8bc422a1/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:54.636568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.644091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 913/1830] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:54 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:54.645023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.783291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31eeb5-8314-4694-8d4e-18cbf564c5cb admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.829814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63ccf-0874-4b8d-b334-7263ac582991 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:54.875898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.877215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.879073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.879647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.880318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.882647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:54.943329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:54.948037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 914/1831] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:54 2022] POST /identity/v3/projects => generated 385 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:54.956600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:54.967650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:54.968291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:54.968573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:54.968848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:54.969499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:54.971868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.030740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 918/1832] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:54 2022] POST /identity/v3/projects => generated 391 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:55.031746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.096005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc4b41-48de-470d-90db-3d321855639e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.182889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e830f5-46a5-48a4-8c16-d1b0161f4391 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.194918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.195590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.195838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.196140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.201384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.205697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.208351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:55.232675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.234064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.234755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.235370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.236403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.238944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.257527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 915/1833] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:55.259004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.265000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 919/1834] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:55 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:55.266003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.284070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc02ff8-240d-4921-8566-a5490ecab2c7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.289342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a75b3-29d2-4bca-b35c-a9bc80d0a72c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.297780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.298470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.298727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.299154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.299809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.300135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.300392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.301232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.301346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.301704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.305131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.305285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.357659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 920/1835] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:55 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:55.371008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.392960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25619b65-4f85-46a8-9f25-892f00f97af0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.396624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 916/1836] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:55 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:55.403420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.418676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.419703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.419703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.419944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] PATH_INFO: `/v3/projects/58b6ff2e844e4041a64946b89fe5ff29/users/006e8964a50a417d8d5cda9d8bc422a1/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.430504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a708d0-3f0e-48ef-863f-9c4a0396a253 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.433293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b6ff2e844e4041a64946b89fe5ff29, user_id=006e8964a50a417d8d5cda9d8bc422a1, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.437938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.451799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.452695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.453070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.453472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.454670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.462537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.463852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 921/1837] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:55 2022] PUT /identity/v3/projects/58b6ff2e844e4041a64946b89fe5ff29/users/006e8964a50a417d8d5cda9d8bc422a1/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:55.464743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.507328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 917/1838] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:55.507328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.518147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bb570-8703-4e78-a855-67b413290404 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.610961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd2966-3b8b-41fe-99d3-174aaaaf12b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.612129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd2966-3b8b-41fe-99d3-174aaaaf12b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.612488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd2966-3b8b-41fe-99d3-174aaaaf12b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.612941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd2966-3b8b-41fe-99d3-174aaaaf12b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.613675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd2966-3b8b-41fe-99d3-174aaaaf12b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.615532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd2966-3b8b-41fe-99d3-174aaaaf12b0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.621814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f969-086e-4492-8c04-43b06993e3ff admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.627947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.628119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 918/1839] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:55.629072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.629500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.629933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.630044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.630469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.630969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.631559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.633513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.653754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 922/1840] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:55.655385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.672012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cd509-d479-4749-a784-d4f4da8ab0a4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.677129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] MFA Rules not processed for user `006e8964a50a417d8d5cda9d8bc422a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:55.678586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:55.678992 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69547a-baa6-4407-ad07-f43663f9359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.679869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69547a-baa6-4407-ad07-f43663f9359f admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.680231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69547a-baa6-4407-ad07-f43663f9359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.680511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69547a-baa6-4407-ad07-f43663f9359f admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.681554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69547a-baa6-4407-ad07-f43663f9359f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.683438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69547a-baa6-4407-ad07-f43663f9359f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.699007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 923/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:55.700641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.700978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.701257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.739193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] MFA Rules not processed for user `2aaa8a7f386e4ccc9ab5c356cdf1704f`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:55.740530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:55.766550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:55.856354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 924/1842] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:55.857605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.882697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 919/1843] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:55.888326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ced44b-a1fe-49ba-97d6-58f7225f905b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.898614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.904874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.905561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.906114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.906759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.908351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:55.909166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:55.921543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278c43c-8a39-4dfc-b4ba-33e05c486933 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:55.934759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 925/1844] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:55.934759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:55.948962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:55.950078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:55.950663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:55.951146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] PATH_INFO: `/v3/projects/0bdb2d44a069400898504c619a04a299/users/ff9502d2e55c40e2b06070a081593969/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:55.984961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd16e82-7000-400d-aca1-87a342a7dd2b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:56.008819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:56.008819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.008819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.008819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] PATH_INFO: `/v3/projects/fab6cfb2433d4672b4a6d30aac1bbcfc/users/040b90d7575b4a538770deb441223064/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.011556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdb2d44a069400898504c619a04a299, user_id=ff9502d2e55c40e2b06070a081593969, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.015944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.047894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 920/1845] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:55 2022] PUT /identity/v3/projects/0bdb2d44a069400898504c619a04a299/users/ff9502d2e55c40e2b06070a081593969/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:56.048766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:56.051809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:56.072007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab6cfb2433d4672b4a6d30aac1bbcfc, user_id=040b90d7575b4a538770deb441223064, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.077231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.088238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:56.108518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 926/1846] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:55 2022] PUT /identity/v3/projects/fab6cfb2433d4672b4a6d30aac1bbcfc/users/040b90d7575b4a538770deb441223064/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:56.110656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:56.192248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70336b9f-07a5-4b7e-ba65-9b9800e4a102 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:56.213904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58c651-b3f0-404e-bf9e-ffdf1c1be084 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:56.229103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:56.230553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.231090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.231709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] PATH_INFO: `/v3/projects/6d7d0244d97f41ef96d4290ce13732a3/users/3018b232a5ae4035ab89ce294189ecdb/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.256403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7d0244d97f41ef96d4290ce13732a3, user_id=3018b232a5ae4035ab89ce294189ecdb, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.260304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.294334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 927/1847] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:56 2022] PUT /identity/v3/projects/6d7d0244d97f41ef96d4290ce13732a3/users/3018b232a5ae4035ab89ce294189ecdb/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:56.295397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:56.319541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:56.322502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.323096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.325372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.328903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:56.475509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e1799-0828-42b0-9641-de6dbd8befd2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:56.488091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86be48-6b04-4658-bc96-eff7d357cd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:56.489589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86be48-6b04-4658-bc96-eff7d357cd32 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.490168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86be48-6b04-4658-bc96-eff7d357cd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.490640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86be48-6b04-4658-bc96-eff7d357cd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.491536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86be48-6b04-4658-bc96-eff7d357cd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.493568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86be48-6b04-4658-bc96-eff7d357cd32 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.502983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.506764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3557ed9-5337-44db-a345-728f9ed14ad6 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.508654 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 928/1848] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:56.510372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661b39-e3a4-4a30-858e-ce8c40e891bf None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.510950 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661b39-e3a4-4a30-858e-ce8c40e891bf None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.511476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661b39-e3a4-4a30-858e-ce8c40e891bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.545462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 921/1849] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:56.547754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.549507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.550237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.568269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:56.575093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-31661b39-e3a4-4a30-858e-ce8c40e891bf None None] MFA Rules not processed for user `040b90d7575b4a538770deb441223064`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:56.578044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-31661b39-e3a4-4a30-858e-ce8c40e891bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:56.646391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] MFA Rules not processed for user `192a8abd4c7a4535a42bea7648535dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:56.647634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:56.718514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 929/1850] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:56.721702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d04699-ad18-4986-a934-b048a739f614 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.722135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d04699-ad18-4986-a934-b048a739f614 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.722547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d04699-ad18-4986-a934-b048a739f614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.775190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-58d04699-ad18-4986-a934-b048a739f614 None None] MFA Rules not processed for user `3018b232a5ae4035ab89ce294189ecdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:56.776480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-58d04699-ad18-4986-a934-b048a739f614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:56.843971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 922/1851] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:56.845142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:56.871371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61194db6-e56f-4da8-9222-aca0602a3f44 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:56.878055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 930/1852] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:56.878055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9596-698f-42d6-a6d4-66c80c083c34 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.878055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9596-698f-42d6-a6d4-66c80c083c34 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.878055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9596-698f-42d6-a6d4-66c80c083c34 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.879228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 931/1853] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:56.880043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9596-698f-42d6-a6d4-66c80c083c34 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:56.894789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9596-698f-42d6-a6d4-66c80c083c34 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:56.895051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:56.898015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.899261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.900689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] PATH_INFO: `/v3/projects/0bdb2d44a069400898504c619a04a299/users/ff9502d2e55c40e2b06070a081593969/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.901600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:56.902383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:56.902805 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:56.903267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:56.904431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.907567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.911899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdb2d44a069400898504c619a04a299, user_id=ff9502d2e55c40e2b06070a081593969, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:56.923212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:56.960515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 932/1855] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:56 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:56.961885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:56.966212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 923/1855] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:56 2022] PUT /identity/v3/projects/0bdb2d44a069400898504c619a04a299/users/ff9502d2e55c40e2b06070a081593969/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:56.967228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.038245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:57.107334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea2ce7-2c93-4283-b593-776e6e83b4b7 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.113927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63a01f-57b3-492f-b41d-e8c3d4b6dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.115103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63a01f-57b3-492f-b41d-e8c3d4b6dcd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.116061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63a01f-57b3-492f-b41d-e8c3d4b6dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.126334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63a01f-57b3-492f-b41d-e8c3d4b6dcd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.127209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63a01f-57b3-492f-b41d-e8c3d4b6dcd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.132166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63a01f-57b3-492f-b41d-e8c3d4b6dcd6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.138487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82af9cd-acf1-4a88-bc31-19cb607f2c68 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.143980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07657a41-6a3e-413b-a7a8-6f319b1edf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.144585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07657a41-6a3e-413b-a7a8-6f319b1edf21 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.145084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07657a41-6a3e-413b-a7a8-6f319b1edf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.145460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07657a41-6a3e-413b-a7a8-6f319b1edf21 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.146197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07657a41-6a3e-413b-a7a8-6f319b1edf21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.148846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07657a41-6a3e-413b-a7a8-6f319b1edf21 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.185433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:57.207699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 924/1856] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:56 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:57.211240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.211549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.212961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.220306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 933/1857] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:40:56 2022] POST /identity/v3/projects => generated 387 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:57.220306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.220306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.220306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.252894 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:57.254214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:57.283488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] MFA Rules not processed for user `ff9502d2e55c40e2b06070a081593969`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:57.284766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:57.373392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 934/1858] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:40:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:57.374699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.466105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 925/1859] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:57.467188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.502172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628386c-3c56-44c8-a949-a130207e9cfb None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.508277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.509322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.509841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.510346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.511455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.513476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.534962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 935/1860] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:57.536245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.603001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7440218-0b19-4c50-a23d-70166f964018 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.615425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.616686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.617121 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.617542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.618364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.620607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.640186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16b4e4-24b3-4c49-992e-f2193be6b556 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.654546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 926/1861] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:40:57 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:40:57.655938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.658645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83781d47-6f15-4cd2-a587-aa0067066d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.659675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83781d47-6f15-4cd2-a587-aa0067066d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.660170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83781d47-6f15-4cd2-a587-aa0067066d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.660704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83781d47-6f15-4cd2-a587-aa0067066d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.672498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83781d47-6f15-4cd2-a587-aa0067066d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.674698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83781d47-6f15-4cd2-a587-aa0067066d34 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.679258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31474ca4-14a8-42e3-9648-35901f3b1be2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.700940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9314e-3351-4e29-861f-20b13a957b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.701261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 936/1862] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:57.701657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9314e-3351-4e29-861f-20b13a957b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.701918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9314e-3351-4e29-861f-20b13a957b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.702214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9314e-3351-4e29-861f-20b13a957b11 admin admin] PATH_INFO: `/v3/projects/f5ccacc299504b4b983ae7ec02d7b478/users/8952f1a931ac471494d400a1f04a609d/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.702456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e5b23-dc17-451d-82ca-b6f1786ae991 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.702801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e5b23-dc17-451d-82ca-b6f1786ae991 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.703279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e5b23-dc17-451d-82ca-b6f1786ae991 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.704955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 937/1863] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:40:57.705748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e5b23-dc17-451d-82ca-b6f1786ae991 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.723123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e5b23-dc17-451d-82ca-b6f1786ae991 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.732176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9314e-3351-4e29-861f-20b13a957b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ccacc299504b4b983ae7ec02d7b478, user_id=8952f1a931ac471494d400a1f04a609d, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.735147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9314e-3351-4e29-861f-20b13a957b11 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.742266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:57.746847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.747529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.747753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.748079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.749092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.753965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.764840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 927/1864] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:57 2022] PUT /identity/v3/projects/f5ccacc299504b4b983ae7ec02d7b478/users/8952f1a931ac471494d400a1f04a609d/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:57.766186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.766482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.766754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.779005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 938/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:57.779878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.798001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca8f3b-d219-489c-8d9f-3919f5046756 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:57.811817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:57.813181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:57.819577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:57.897390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:57.898754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:57.899123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:57.899561 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:57.900444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:57.902728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:57.937208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 939/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:57.937208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:57.975991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7391a6-8b60-4d1b-9677-5dcead0d1aae admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:58.000935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:58.001682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.001967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.002279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] PATH_INFO: `/v3/projects/9760ebba004041b6aeaa45c8bd48700d/users/8b8bb7963f6d4bdda32ad6ff5cdbe663/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.015861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 928/1867] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:40:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:58.016633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.022868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9760ebba004041b6aeaa45c8bd48700d, user_id=8b8bb7963f6d4bdda32ad6ff5cdbe663, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:58.026088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:58.027433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:58.052016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 940/1868] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:57 2022] PUT /identity/v3/projects/9760ebba004041b6aeaa45c8bd48700d/users/8b8bb7963f6d4bdda32ad6ff5cdbe663/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:58.052016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.163684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e190e-5bbb-4b5b-96ea-aaa9fd05d667 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:58.174070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3ed1-78e4-464b-8272-3d22a4b80672 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:58.181168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:58.182284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.183488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.183989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] PATH_INFO: `/v3/projects/f5ccacc299504b4b983ae7ec02d7b478/users/8952f1a931ac471494d400a1f04a609d/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.204121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ccacc299504b4b983ae7ec02d7b478, user_id=8952f1a931ac471494d400a1f04a609d, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:58.211977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:58.248098 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 941/1869] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:58 2022] PUT /identity/v3/projects/f5ccacc299504b4b983ae7ec02d7b478/users/8952f1a931ac471494d400a1f04a609d/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:58.250839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.279143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:58.295446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.296177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.299949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.358032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86656ea-ced5-4ca6-ac7c-22ccf49df884 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:58.380013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d647a12-fe0a-476a-831d-bdb329050275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:58.395569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d647a12-fe0a-476a-831d-bdb329050275 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.395982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d647a12-fe0a-476a-831d-bdb329050275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.396540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d647a12-fe0a-476a-831d-bdb329050275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.397341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d647a12-fe0a-476a-831d-bdb329050275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:58.402587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d647a12-fe0a-476a-831d-bdb329050275 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:58.409241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:58.409241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:58.423519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:58.437650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 942/1870] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:40:58.441324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 929/1871] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:58.442064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.454295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.456690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.457131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.468450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dff1cf-5393-424d-a6a2-1e3c3681563f service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:58.483524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90ad65-b201-4436-87ac-fa811ebecc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:58.484228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90ad65-b201-4436-87ac-fa811ebecc1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.484510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90ad65-b201-4436-87ac-fa811ebecc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.484942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90ad65-b201-4436-87ac-fa811ebecc1a admin admin] PATH_INFO: `/v3/projects/9760ebba004041b6aeaa45c8bd48700d/users/8b8bb7963f6d4bdda32ad6ff5cdbe663/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.491616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90ad65-b201-4436-87ac-fa811ebecc1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9760ebba004041b6aeaa45c8bd48700d, user_id=8b8bb7963f6d4bdda32ad6ff5cdbe663, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:58.495365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90ad65-b201-4436-87ac-fa811ebecc1a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:58.507641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] MFA Rules not processed for user `8952f1a931ac471494d400a1f04a609d`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:58.509552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:58.521599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 930/1872] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:40:58 2022] PUT /identity/v3/projects/9760ebba004041b6aeaa45c8bd48700d/users/8b8bb7963f6d4bdda32ad6ff5cdbe663/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:58.536738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.537680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.538120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.541444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:58.615262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] MFA Rules not processed for user `8b8bb7963f6d4bdda32ad6ff5cdbe663`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:58.618667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:58.668154 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 943/1873] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:58.760099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.856201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8399-4343-46be-96b9-2bee8af3f76a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:58.860826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:58.863086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:58.863444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:58.863703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:58.873181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 931/1874] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:58.883325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:58.888357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:58.907335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 944/1875] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:58.965590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.975160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:58.989993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c89a7-5649-465b-afa9-f6e587d7859c service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:59.009443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86b298-4bb5-476a-9648-84d470e5d441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:59.013373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86b298-4bb5-476a-9648-84d470e5d441 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:59.013768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86b298-4bb5-476a-9648-84d470e5d441 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:59.014725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86b298-4bb5-476a-9648-84d470e5d441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:59.034917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86b298-4bb5-476a-9648-84d470e5d441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:59.036743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86b298-4bb5-476a-9648-84d470e5d441 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:59.052389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 945/1876] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:59.141061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f9a23-9b08-49ed-b787-02ca995e607b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:59.147416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:59.150029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:59.150397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:59.150937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] PATH_INFO: `/v3/users/040b90d7575b4a538770deb441223064` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:59.152934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=040b90d7575b4a538770deb441223064)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:59.155313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:59.274474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '040b90d7575b4a538770deb441223064'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:59.275220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '040b90d7575b4a538770deb441223064'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:59.291543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '040b90d7575b4a538770deb441223064'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:59.301754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '040b90d7575b4a538770deb441223064'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:59.311573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:59.342018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '040b90d7575b4a538770deb441223064'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:40:59.360429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 932/1877] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:40:58 2022] DELETE /identity/v3/users/040b90d7575b4a538770deb441223064 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:40:59.366047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:59.367067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:59.367530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:59.401044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:59.402723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:59.422342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:40:59.423738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:59.543162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 946/1878] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:40:59.557577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:59.559415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:59.620190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:40:59.634846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9243f7-5914-4ac5-88a7-0c787bec1ad6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:59.642689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:59.643999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:59.644382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:59.644541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:59.647438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:59.661535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:59.663463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:59.664847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ca81d-4a42-4121-aae2-902257f87916 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:59.671133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:59.672581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:59.673360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:59.673972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:59.677403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 933/1879] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:40:59.678586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:40:59.680037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:59.687159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:59.689959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:59.705774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d86b76-dbec-4539-b104-3a5353f665db service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:40:59.707138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 947/1880] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:40:59.733079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:40:59.733675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:40:59.733921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:40:59.734208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:40:59.739289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:40:59.774109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:40:59.777486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:40:59.806618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 934/1881] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:40:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:00.102282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:00.104139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.119514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cafa8d-16e3-4493-8429-768ed585f266 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:00.125389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:00.126373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:00.127124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:00.127615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] PATH_INFO: `/v3/users/3018b232a5ae4035ab89ce294189ecdb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:00.129245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3018b232a5ae4035ab89ce294189ecdb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:00.132645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:00.162339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:00.164491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.187567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e9aff-6f17-4a71-a20e-36d94bdeba88 service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:00.194602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:00.195317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:00.195641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:00.195917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] PATH_INFO: `/v3/projects/fab6cfb2433d4672b4a6d30aac1bbcfc` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:00.197885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab6cfb2433d4672b4a6d30aac1bbcfc)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:00.199809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:00.226529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3018b232a5ae4035ab89ce294189ecdb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.236066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3018b232a5ae4035ab89ce294189ecdb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.236704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3018b232a5ae4035ab89ce294189ecdb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.244636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3018b232a5ae4035ab89ce294189ecdb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.257451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6430927-a0c3-44f8-aec3-e0cd99e2f707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3018b232a5ae4035ab89ce294189ecdb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.263918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 948/1882] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:00 2022] DELETE /identity/v3/users/3018b232a5ae4035ab89ce294189ecdb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:00.264900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab6cfb2433d4672b4a6d30aac1bbcfc', 'request_id': 'req-72367931-527b-4d7c-bb7e-d94154b42b65'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.268210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:00.268495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:00.268796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:00.271532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab6cfb2433d4672b4a6d30aac1bbcfc', 'request_id': 'req-72367931-527b-4d7c-bb7e-d94154b42b65'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.277058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab6cfb2433d4672b4a6d30aac1bbcfc', 'request_id': 'req-72367931-527b-4d7c-bb7e-d94154b42b65'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.287349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:00.288585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.290576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] The token cache is being invalidate because project fab6cfb2433d4672b4a6d30aac1bbcfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:00.290871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.291483 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.293035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 935/1883] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:00 2022] DELETE /identity/v3/projects/fab6cfb2433d4672b4a6d30aac1bbcfc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:00.297063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:00.298420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.343213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:00.348769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 949/1884] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:00.359644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:00.361294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.365122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72367931-527b-4d7c-bb7e-d94154b42b65 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:00.369734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:00.370343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:00.370625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:00.370892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] PATH_INFO: `/v3/users/8952f1a931ac471494d400a1f04a609d` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:00.372021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8952f1a931ac471494d400a1f04a609d)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:00.373941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:00.417670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:00.422608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53116815-1ec2-40d5-b37a-ac353b6c8516 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:00.428390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:00.429088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:00.429382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:00.429929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:00.440761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8952f1a931ac471494d400a1f04a609d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.442613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:00.444078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:00.444357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8952f1a931ac471494d400a1f04a609d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.454082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8952f1a931ac471494d400a1f04a609d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.455605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 950/1885] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:00.465649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8952f1a931ac471494d400a1f04a609d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.480381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8952f1a931ac471494d400a1f04a609d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:00.487002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 936/1886] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:00 2022] DELETE /identity/v3/users/8952f1a931ac471494d400a1f04a609d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:00.887407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:00.888857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.901962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020f9ca-a60c-403e-817a-2cbead3e001e service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:00.943256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:00.945463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:00.973453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:00.974391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:00.974820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:00.975157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] PATH_INFO: `/v3/projects/6d7d0244d97f41ef96d4290ce13732a3` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:00.976376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7d0244d97f41ef96d4290ce13732a3)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:00.978886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.038352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7d0244d97f41ef96d4290ce13732a3', 'request_id': 'req-6ec76d7f-3a63-409a-b521-22bd3dcfc753'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.048185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7d0244d97f41ef96d4290ce13732a3', 'request_id': 'req-6ec76d7f-3a63-409a-b521-22bd3dcfc753'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.054470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270c9a9-6e13-4fbe-b306-d5f9a812d082 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.055947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7d0244d97f41ef96d4290ce13732a3', 'request_id': 'req-6ec76d7f-3a63-409a-b521-22bd3dcfc753'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.066896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] The token cache is being invalidate because project 6d7d0244d97f41ef96d4290ce13732a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:01.067478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.071785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.077198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 951/1887] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:00 2022] DELETE /identity/v3/projects/6d7d0244d97f41ef96d4290ce13732a3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:01.079160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.080287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.130674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.131353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.131635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.131911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] PATH_INFO: `/v3/projects/f5ccacc299504b4b983ae7ec02d7b478` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.133576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ccacc299504b4b983ae7ec02d7b478)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.135527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.148319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:01.152422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76d7f-3a63-409a-b521-22bd3dcfc753 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.157480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.158140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.158490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.158729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] PATH_INFO: `/v3/users/8b8bb7963f6d4bdda32ad6ff5cdbe663` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.159975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b8bb7963f6d4bdda32ad6ff5cdbe663)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.161933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.186746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:01.204367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ccacc299504b4b983ae7ec02d7b478', 'request_id': 'req-6d36829a-0742-4411-a04f-a912bf350cb9'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.213869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ccacc299504b4b983ae7ec02d7b478', 'request_id': 'req-6d36829a-0742-4411-a04f-a912bf350cb9'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.223032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ccacc299504b4b983ae7ec02d7b478', 'request_id': 'req-6d36829a-0742-4411-a04f-a912bf350cb9'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.238451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b8bb7963f6d4bdda32ad6ff5cdbe663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.245056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] The token cache is being invalidate because project f5ccacc299504b4b983ae7ec02d7b478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:01.246061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.246918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d36829a-0742-4411-a04f-a912bf350cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.248892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b8bb7963f6d4bdda32ad6ff5cdbe663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.249858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b8bb7963f6d4bdda32ad6ff5cdbe663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.252792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 937/1888] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:00 2022] DELETE /identity/v3/projects/f5ccacc299504b4b983ae7ec02d7b478 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:01.254314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.254577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.254865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.259263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b8bb7963f6d4bdda32ad6ff5cdbe663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.272994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c3a4a5b5-0f7a-4af6-ab1d-28e07011d307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b8bb7963f6d4bdda32ad6ff5cdbe663'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.284265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 952/1889] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:01 2022] DELETE /identity/v3/users/8b8bb7963f6d4bdda32ad6ff5cdbe663 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:01.291939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:01.293406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.378440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 938/1890] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:01.386403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.386636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.386913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.407422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:01.408820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.417288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 953/1891] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:01.421612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.431716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1492bf8-1e3b-4ce1-9e1e-800e10a1e548 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.437803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.438478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.438770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.439047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.439712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:01.440017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.441743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.454470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 939/1892] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:01.458749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.468393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cd314-bac3-4f46-868a-0fbdb5db71e2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.474014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.474827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.475099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.475357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.476021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.477918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.508585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 954/1893] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:01.512884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.523253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5c7f-2fe3-45f0-8b1e-9a5457d70c37 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.571438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.573529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.583641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.584280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.585187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.585187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.585451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.597464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.631126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 940/1894] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:01.635977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.649372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec818-af53-4759-a06b-08dd90bfce71 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.659784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d25434-bc0b-4538-94e3-dadb991ff2e8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.666962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.668616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.669002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.669378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] PATH_INFO: `/v3/projects/9760ebba004041b6aeaa45c8bd48700d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.671330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9760ebba004041b6aeaa45c8bd48700d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.673946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.674432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.675144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.675432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.675757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.676489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.678752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.719134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 941/1895] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:01.720180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.733500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855dad-08ee-4797-bccc-986521321ccb admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.738201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:01.758290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9760ebba004041b6aeaa45c8bd48700d', 'request_id': 'req-bf9ca8a6-4a53-4581-8559-833f15e39d9e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.774798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9760ebba004041b6aeaa45c8bd48700d', 'request_id': 'req-bf9ca8a6-4a53-4581-8559-833f15e39d9e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.785004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9760ebba004041b6aeaa45c8bd48700d', 'request_id': 'req-bf9ca8a6-4a53-4581-8559-833f15e39d9e'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.813016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.813643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.813967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.814221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.814985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.816735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.818240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] The token cache is being invalidate because project 9760ebba004041b6aeaa45c8bd48700d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:01.818240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.818962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bf9ca8a6-4a53-4581-8559-833f15e39d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:01.820714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 955/1896] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:01 2022] DELETE /identity/v3/projects/9760ebba004041b6aeaa45c8bd48700d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:01.826454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5da63-17c9-4593-b9ae-d1d00c06f319 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.826770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5da63-17c9-4593-b9ae-d1d00c06f319 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.827132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5da63-17c9-4593-b9ae-d1d00c06f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.830566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 942/1897] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:01.834504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.836505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.838814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:01.851704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a5d5f-475d-4b0f-a0e6-d19623be1bf1 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.853884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bfb5da63-17c9-4593-b9ae-d1d00c06f319 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:01.857622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c892f-2592-4d6d-bb0f-91d0df6c167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.858135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bfb5da63-17c9-4593-b9ae-d1d00c06f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.858385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c892f-2592-4d6d-bb0f-91d0df6c167d admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.858615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c892f-2592-4d6d-bb0f-91d0df6c167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.858886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c892f-2592-4d6d-bb0f-91d0df6c167d admin admin] PATH_INFO: `/v3/projects/c1e53b310c9c4b3baba0b8c8501d0481/users/56844f1a822547cbbbf59e217d749240/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.868086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 956/1898] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:01.876217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.876583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.877020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.881242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c892f-2592-4d6d-bb0f-91d0df6c167d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e53b310c9c4b3baba0b8c8501d0481, user_id=56844f1a822547cbbbf59e217d749240, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.883957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c892f-2592-4d6d-bb0f-91d0df6c167d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:01.900752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 943/1899] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:01 2022] PUT /identity/v3/projects/c1e53b310c9c4b3baba0b8c8501d0481/users/56844f1a822547cbbbf59e217d749240/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:01.901100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:01.902510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.911037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.911322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.911615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.939986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] MFA Rules not processed for user `56844f1a822547cbbbf59e217d749240`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:01.941242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:01.972251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 957/1900] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:01.976467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:01.985874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab57e05-748c-4b3c-8827-ca80490a219f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:01.994839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:01.996074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:01.996360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:01.996640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:01.997355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:01.997650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:01.999337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.012045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 958/1901] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:41:02.017502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 944/1902] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:02.018639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.021890 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.029436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dda2f7-734a-4ce3-9c0e-7da9179eb304 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.032980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c4bd2-382a-4a4c-a150-5aff513340e6 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.035114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.035827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.036415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.036617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.037244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.038035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.038680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.038959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.039606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.039724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.039895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.041722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.071852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 959/1903] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:02 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:02.076980 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.081005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 945/1904] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:02 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:02.082050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.149492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:02.163765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b66d-2a65-4567-bebe-824ff270b3a1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.166808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7dd222-0831-42c4-a589-f2acb5065743 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.173300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.174347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.174558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.174659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.174659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.174893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.175193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.175418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.175511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.176566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.177846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.179305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.207994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 946/1905] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:02 2022] POST /identity/v3/users => generated 306 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:02.207994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.210556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:02.211767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 960/1906] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:02 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:02.212796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.247600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc03f5-8b15-44d8-85b8-adb0dba48bf3 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.268158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.270501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.270858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.271060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.271746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.273523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.285307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 961/1907] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:02.286312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.301076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b7372-b8b5-43c8-8fea-b9dc268989a8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.307706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.308688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.309042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.309389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.310257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.312388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.328356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 962/1908] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:02.330413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.345233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfd760-1bcf-44ce-9c82-e83e49ebafeb admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.348736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58dcc9-f18f-4082-8610-10abe52d3fb5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.351122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.351791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.352073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.352348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] PATH_INFO: `/v3/projects/2b4068468a414a44ad295266d3bb21f0/users/ff7c98ff63044b1d9435c9ed5c1ad8e9/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.357964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.358634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.358891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.359161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.362004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:02.377874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4068468a414a44ad295266d3bb21f0, user_id=ff7c98ff63044b1d9435c9ed5c1ad8e9, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.394040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.425482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 963/1909] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:02 2022] PUT /identity/v3/projects/2b4068468a414a44ad295266d3bb21f0/users/ff7c98ff63044b1d9435c9ed5c1ad8e9/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:02.426484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.434173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:02.490292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.493009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.510664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 947/1910] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:02.511475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.524831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accc18f-5b16-4278-9431-ba0dd87293d4 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.590442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc5109-1566-4689-9023-9c22aacfef6a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.607881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.611312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.611722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.611922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.616755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.622470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.637934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.639196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.639615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.650343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 964/1911] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:02.650807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.651285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.652706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.660125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.676868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a7fb5-a68d-451c-b45f-dbabd62ebe2d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.687086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854461f-30ef-4e1d-a617-c2e92b739902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.687955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854461f-30ef-4e1d-a617-c2e92b739902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.688267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854461f-30ef-4e1d-a617-c2e92b739902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.688537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854461f-30ef-4e1d-a617-c2e92b739902 admin admin] PATH_INFO: `/v3/projects/1c67d77e94db4a64b2c67529ad937773/users/d91af76d529d4a60a1fa34a6246d6691/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.693237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 948/1912] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:02.694141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:02.726235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc103bf5-31d0-4d8a-903c-30f962993777 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:02.727581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854461f-30ef-4e1d-a617-c2e92b739902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c67d77e94db4a64b2c67529ad937773, user_id=d91af76d529d4a60a1fa34a6246d6691, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.730910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854461f-30ef-4e1d-a617-c2e92b739902 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.738597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3460e2-8362-49a8-917c-3d6a1e95f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:02.739445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3460e2-8362-49a8-917c-3d6a1e95f2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.739749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3460e2-8362-49a8-917c-3d6a1e95f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.740002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3460e2-8362-49a8-917c-3d6a1e95f2d2 admin admin] PATH_INFO: `/v3/projects/2b4068468a414a44ad295266d3bb21f0/users/ff7c98ff63044b1d9435c9ed5c1ad8e9/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.742141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3460e2-8362-49a8-917c-3d6a1e95f2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4068468a414a44ad295266d3bb21f0, user_id=ff7c98ff63044b1d9435c9ed5c1ad8e9, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:02.744801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3460e2-8362-49a8-917c-3d6a1e95f2d2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:02.771509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 949/1913] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:02 2022] PUT /identity/v3/projects/2b4068468a414a44ad295266d3bb21f0/users/ff7c98ff63044b1d9435c9ed5c1ad8e9/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:02.782727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 965/1914] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:02 2022] PUT /identity/v3/projects/1c67d77e94db4a64b2c67529ad937773/users/d91af76d529d4a60a1fa34a6246d6691/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:02.798090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.798622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:02.798990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.799301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.800608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:02.800894 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:02.866631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:02.893695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] MFA Rules not processed for user `ff7c98ff63044b1d9435c9ed5c1ad8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:02.895262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:02.903760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] MFA Rules not processed for user `d91af76d529d4a60a1fa34a6246d6691`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:02.905165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:03.040130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 950/1915] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:03.066495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 966/1916] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:03.067490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.088872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.090382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:03.215685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98832af3-012c-4ec0-9382-7efa3b5ad3da None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.223372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.224301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.224671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.225306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.259165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125cc9a-10e5-4fb1-8829-1c4e98e8a935 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.265965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.270022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.275755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.277376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.279460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.280100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.282043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.286760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.303885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 967/1917] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:03.370412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 951/1918] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:03 2022] POST /identity/v3/projects => generated 381 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:03.374434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.389076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e1e-8b3a-4b25-83e0-a8fa489a11a1 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.404089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:03.493089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.538058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.538879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.539272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.539644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.540529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.542775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.575308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef8bc2-ef1e-4493-bc64-271d947770f5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.579373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 968/1919] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:03 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:03.582019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.583014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.583298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.583568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.584552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.598967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569593-7dbe-410e-b11e-d1227b181bfd admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.604474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.605866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.606291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.606665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.607538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.609598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.623998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 969/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:03.628257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.630658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:03.646532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22ce8-cdd9-4618-ba13-7edc81afd890 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.655638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.657501 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.657888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.658335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] PATH_INFO: `/v3/projects/e20908313ba14396b3f878fcefc1073e/users/01700081b83c42be955516708d057f29/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.670319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.672044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.683714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20908313ba14396b3f878fcefc1073e, user_id=01700081b83c42be955516708d057f29, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.686760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.694484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 952/1921] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:03.711760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 970/1922] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:03 2022] PUT /identity/v3/projects/e20908313ba14396b3f878fcefc1073e/users/01700081b83c42be955516708d057f29/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:03.715538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.729939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e3c6-ee46-4075-8f5a-e3a71d3edeb3 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.824937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756da65-90a4-407a-a186-cdbedf207729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.826011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756da65-90a4-407a-a186-cdbedf207729 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.826416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756da65-90a4-407a-a186-cdbedf207729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.826862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756da65-90a4-407a-a186-cdbedf207729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.828019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756da65-90a4-407a-a186-cdbedf207729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.829649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756da65-90a4-407a-a186-cdbedf207729 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.844110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 953/1923] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:03.845676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:03.868882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b397e0e-1812-4a5d-88c1-5d5570741551 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:03.892249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:03.892987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.893708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.894340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] PATH_INFO: `/v3/projects/e20908313ba14396b3f878fcefc1073e/users/01700081b83c42be955516708d057f29/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.899219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20908313ba14396b3f878fcefc1073e, user_id=01700081b83c42be955516708d057f29, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:03.903026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:03.919120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 971/1924] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:03 2022] PUT /identity/v3/projects/e20908313ba14396b3f878fcefc1073e/users/01700081b83c42be955516708d057f29/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:03.933633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:03.934013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:03.934261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:03.984237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] MFA Rules not processed for user `01700081b83c42be955516708d057f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:03.987245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:04.114083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 954/1925] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:04.126540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:04.190543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b0947-f7c5-489a-875d-bb721c2671c1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:04.194294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:04.194917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:04.195199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:04.195474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:04.204885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:04.206447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:04.216647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 972/1926] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:04.311854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:04.321685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5728e9-5f66-48a3-a5b7-809c474bb81e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:04.325384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc565d2-82aa-44f5-8f55-1becfd6d25be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:04.325992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc565d2-82aa-44f5-8f55-1becfd6d25be service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:04.326308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc565d2-82aa-44f5-8f55-1becfd6d25be service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:04.326583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc565d2-82aa-44f5-8f55-1becfd6d25be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:04.384745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:04.389575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc565d2-82aa-44f5-8f55-1becfd6d25be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:04.391379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc565d2-82aa-44f5-8f55-1becfd6d25be service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:04.400936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 955/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:04.681474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:04.735806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:04.748434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b1b0d-f0e3-435a-ba47-0d492e538ec7 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:04.753384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:04.754037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:04.754369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:04.754646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] PATH_INFO: `/v3/users/d91af76d529d4a60a1fa34a6246d6691` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:04.755815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91af76d529d4a60a1fa34a6246d6691)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:04.758045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:04.806010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91af76d529d4a60a1fa34a6246d6691'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:04.815596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91af76d529d4a60a1fa34a6246d6691'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:04.816318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91af76d529d4a60a1fa34a6246d6691'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:04.823704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91af76d529d4a60a1fa34a6246d6691'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:04.837102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91af76d529d4a60a1fa34a6246d6691'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:04.843750 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 973/1928] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:04 2022] DELETE /identity/v3/users/d91af76d529d4a60a1fa34a6246d6691 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:04.848439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:04.848760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:04.849038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:04.869307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:04.870547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:04.936535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 956/1929] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:04.948728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:04.951592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:05.018143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a40a70-3d68-4979-914d-1c394aad35d8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:05.021841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:05.022489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:05.022769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:05.023039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:05.032079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:05.033530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:05.043497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 974/1930] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:05.389354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:05.391053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:05.411561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c7dc-89be-4f0f-9c23-98f7b2d0a3d1 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:05.418523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:05.419304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:05.419591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:05.419872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] PATH_INFO: `/v3/projects/1c67d77e94db4a64b2c67529ad937773` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:05.421115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c67d77e94db4a64b2c67529ad937773)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:05.423017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:05.479810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c67d77e94db4a64b2c67529ad937773', 'request_id': 'req-39e7fce1-806d-4334-8885-e754fce55b77'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.485957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c67d77e94db4a64b2c67529ad937773', 'request_id': 'req-39e7fce1-806d-4334-8885-e754fce55b77'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.491548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c67d77e94db4a64b2c67529ad937773', 'request_id': 'req-39e7fce1-806d-4334-8885-e754fce55b77'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.504398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] The token cache is being invalidate because project 1c67d77e94db4a64b2c67529ad937773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:05.504689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.505317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.506855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 957/1931] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:05 2022] DELETE /identity/v3/projects/1c67d77e94db4a64b2c67529ad937773 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:05.511226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:05.521167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561cb7-c07b-4ba8-b94f-064d1fd6ef67 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:05.586202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:05.586862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:05.587142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:05.587419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] PATH_INFO: `/v3/users/01700081b83c42be955516708d057f29` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:05.588700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=01700081b83c42be955516708d057f29)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:05.590727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:05.615038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:05.654146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01700081b83c42be955516708d057f29'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.662983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01700081b83c42be955516708d057f29'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.663586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01700081b83c42be955516708d057f29'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.670905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01700081b83c42be955516708d057f29'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.685393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9f04984e-9e32-4b86-81a1-a86dc39f7bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01700081b83c42be955516708d057f29'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:05.693891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 975/1932] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:05 2022] DELETE /identity/v3/users/01700081b83c42be955516708d057f29 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:05.985945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:05.987556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:06.000976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7fce1-806d-4334-8885-e754fce55b77 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.064289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.064933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.065217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.065491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] PATH_INFO: `/v3/projects/e20908313ba14396b3f878fcefc1073e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.066683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20908313ba14396b3f878fcefc1073e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.068605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.089876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:06.123006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20908313ba14396b3f878fcefc1073e', 'request_id': 'req-d12cce8d-4813-4dd4-acfc-463af04684c2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:06.131887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20908313ba14396b3f878fcefc1073e', 'request_id': 'req-d12cce8d-4813-4dd4-acfc-463af04684c2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:06.143117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20908313ba14396b3f878fcefc1073e', 'request_id': 'req-d12cce8d-4813-4dd4-acfc-463af04684c2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:06.168277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] The token cache is being invalidate because project e20908313ba14396b3f878fcefc1073e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:06.168570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:06.172362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d12cce8d-4813-4dd4-acfc-463af04684c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:06.177284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 958/1933] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:05 2022] DELETE /identity/v3/projects/e20908313ba14396b3f878fcefc1073e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:06.182134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.183981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.183981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.219865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:06.221436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:06.309692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 976/1934] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:06.317959 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.318398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.319069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.353404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:06.355100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:06.363920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 959/1935] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:06.368287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.378847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c51c1-fba2-4073-b33f-81919cd3c88f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.383152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.384220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.384426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.384710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.385467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:06.385856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.387675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.405630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 977/1936] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:06.406893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.420888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d824-ce27-404a-8a33-4130b219a2bf None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.438615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.439315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.439625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.439996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.440699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.442692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.475686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 960/1937] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:06 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:06.476827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.488110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0d6b-bf0f-4811-8ce7-88ca13e366ce admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.604557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.605297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.605585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.605859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.606814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.609292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.648090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 978/1938] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:06 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:06.659212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.672280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59239a18-1529-4113-a4bc-ba09cf9662d9 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.680289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.681069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.681354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.681770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.682529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.684369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.695347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 961/1939] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:06.699502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.710961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46594d2b-99c1-4d2c-9caa-a0194d8dac60 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.717483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3df52-0383-460a-b392-60ce9fe045c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.718123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3df52-0383-460a-b392-60ce9fe045c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.718570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3df52-0383-460a-b392-60ce9fe045c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.720550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3df52-0383-460a-b392-60ce9fe045c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.721397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3df52-0383-460a-b392-60ce9fe045c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.723781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3df52-0383-460a-b392-60ce9fe045c5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.739080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:06.746120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 979/1940] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:06.750346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.761216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9ab21-3eee-4c71-a58b-2b43de4c6c0c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:06.766435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:06.767076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.767362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.770778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] PATH_INFO: `/v3/projects/5f60e8264ff2419f9fc7bc6f3aaaa940/users/fd0471f8c09945f4b99f7ba16d5e3b47/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.791047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f60e8264ff2419f9fc7bc6f3aaaa940, user_id=fd0471f8c09945f4b99f7ba16d5e3b47, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:06.798041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:06.812053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 962/1941] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:06 2022] PUT /identity/v3/projects/5f60e8264ff2419f9fc7bc6f3aaaa940/users/fd0471f8c09945f4b99f7ba16d5e3b47/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:06.823574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:06.824181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:06.824458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:06.854521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] MFA Rules not processed for user `fd0471f8c09945f4b99f7ba16d5e3b47`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:06.855786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:06.923274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 980/1942] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:06.935331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:06.937107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:07.020420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd899b4a-6a11-4ea6-b910-e3323da4ef8b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:07.024156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19bb04-3ea1-44fe-bba5-b469aefdd56c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:07.024860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19bb04-3ea1-44fe-bba5-b469aefdd56c service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:07.025206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19bb04-3ea1-44fe-bba5-b469aefdd56c service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:07.025567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19bb04-3ea1-44fe-bba5-b469aefdd56c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:07.036636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19bb04-3ea1-44fe-bba5-b469aefdd56c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:07.038716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19bb04-3ea1-44fe-bba5-b469aefdd56c service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:07.051269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 963/1943] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:07.075127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:07.159784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22f337-2efa-4b96-bf93-fd936e88c3c5 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:07.168223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:07.169253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:07.169679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:07.170094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] PATH_INFO: `/v3/users/fd0471f8c09945f4b99f7ba16d5e3b47` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:07.171528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0471f8c09945f4b99f7ba16d5e3b47)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:07.173841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:07.284830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0471f8c09945f4b99f7ba16d5e3b47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.302183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0471f8c09945f4b99f7ba16d5e3b47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.302918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0471f8c09945f4b99f7ba16d5e3b47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.313248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0471f8c09945f4b99f7ba16d5e3b47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.348678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0471f8c09945f4b99f7ba16d5e3b47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.376470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 981/1944] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:07 2022] DELETE /identity/v3/users/fd0471f8c09945f4b99f7ba16d5e3b47 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:07.381035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:07.381325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:07.381595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:07.419521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:07.419521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:07.510131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:07.544915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 964/1945] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:07.566968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:07.568829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:07.587232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:07.712548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447830c8-a2f5-4365-b5ce-0c2dad43261e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:07.724879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419db1c3-87c5-4880-adad-775540fd0805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:07.724879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db1c3-87c5-4880-adad-775540fd0805 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:07.724879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db1c3-87c5-4880-adad-775540fd0805 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:07.724879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db1c3-87c5-4880-adad-775540fd0805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:07.726387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:07.728933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:07.756853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0194e5e-0ad4-4c63-ae23-22c27a996e3f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:07.762520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419db1c3-87c5-4880-adad-775540fd0805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:07.768794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419db1c3-87c5-4880-adad-775540fd0805 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:07.779239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:07.779968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:07.781626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:07.781914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] PATH_INFO: `/v3/users/ce841a25351d4679a03205fbb66d5f93` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:07.786699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce841a25351d4679a03205fbb66d5f93)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:07.790856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:07.822985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 982/1946] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:07.925531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce841a25351d4679a03205fbb66d5f93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.926439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce841a25351d4679a03205fbb66d5f93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.952767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce841a25351d4679a03205fbb66d5f93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.963737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce841a25351d4679a03205fbb66d5f93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.985140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce841a25351d4679a03205fbb66d5f93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:07.997117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 965/1947] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:07 2022] DELETE /identity/v3/users/ce841a25351d4679a03205fbb66d5f93 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:08.002380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:08.002909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:08.016547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:08.055266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:08.057220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:08.178221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 983/1948] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:08.260591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:08.262131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:08.429068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a2e47-d7dc-4891-b92e-972c36437b5e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:08.443477 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c1e7b-93fe-45fd-a981-42fb2df7b599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:08.444464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c1e7b-93fe-45fd-a981-42fb2df7b599 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:08.444841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c1e7b-93fe-45fd-a981-42fb2df7b599 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:08.445069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c1e7b-93fe-45fd-a981-42fb2df7b599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:08.471930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c1e7b-93fe-45fd-a981-42fb2df7b599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:08.473455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c1e7b-93fe-45fd-a981-42fb2df7b599 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:08.500682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 966/1949] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:08.975675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:08.978112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:08.993548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cd591-41be-4f2e-92f3-725049d80ffc None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:08.998844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:08.999823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:08.999935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.000322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] PATH_INFO: `/v3/projects/5f60e8264ff2419f9fc7bc6f3aaaa940` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.001448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f60e8264ff2419f9fc7bc6f3aaaa940)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.003596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.068407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f60e8264ff2419f9fc7bc6f3aaaa940', 'request_id': 'req-0d911789-ccac-4b32-98bc-9c28091b6251'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.084339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f60e8264ff2419f9fc7bc6f3aaaa940', 'request_id': 'req-0d911789-ccac-4b32-98bc-9c28091b6251'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.097627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f60e8264ff2419f9fc7bc6f3aaaa940', 'request_id': 'req-0d911789-ccac-4b32-98bc-9c28091b6251'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.115522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] The token cache is being invalidate because project 5f60e8264ff2419f9fc7bc6f3aaaa940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:09.115522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.116168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0d911789-ccac-4b32-98bc-9c28091b6251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.119680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 984/1950] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:08 2022] DELETE /identity/v3/projects/5f60e8264ff2419f9fc7bc6f3aaaa940 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:09.124492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.124790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.125094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.146980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:09.148218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:09.255229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 967/1951] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:09.263502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.263892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.264270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.286951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.290048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:09.303208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:09.313652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:09.314962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:09.322863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23677ba-f901-4024-baf3-7013b9abd9f8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.334234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 985/1952] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:09.335645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.347162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841faaba-0b11-4d7f-aa6f-43d2a274504b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.350093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.350911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.351459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.351757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] PATH_INFO: `/v3/projects/39ce31bd95aa4be1ae98f616475afe83` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.353286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.354451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.354890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.355385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.355490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ce31bd95aa4be1ae98f616475afe83)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.356299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:09.356744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.358531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.362200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.376326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 986/1953] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:09.380756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.402820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a234b-7173-45ec-b75d-28a9e450adc6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.424838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.425553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.425865 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.426097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.426772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.428735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.447118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:09.475508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 987/1954] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:09 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:09.480419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.506070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ce31bd95aa4be1ae98f616475afe83', 'request_id': 'req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.517971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ce31bd95aa4be1ae98f616475afe83', 'request_id': 'req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.527058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ce31bd95aa4be1ae98f616475afe83', 'request_id': 'req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.548293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] The token cache is being invalidate because project 39ce31bd95aa4be1ae98f616475afe83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:09.548633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.549408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.551603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 968/1955] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:09 2022] DELETE /identity/v3/projects/39ce31bd95aa4be1ae98f616475afe83 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:09.556064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.558790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:09.593897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d6b9-8e28-4eef-a912-2acc77b8f31f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.647799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cbc79-88ed-492f-b2b8-0277122ed6f1 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.652528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.653170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.653454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.653726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] PATH_INFO: `/v3/users/ff9502d2e55c40e2b06070a081593969` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.654886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9502d2e55c40e2b06070a081593969)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.656800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.661082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.661682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.661971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.662329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.663183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.665225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.692745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 988/1956] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:09 2022] POST /identity/v3/users => generated 279 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:09.697356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.700004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:09.715380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28979a3f-1c75-4a09-ad6c-b7383a6b1fff admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.720522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9502d2e55c40e2b06070a081593969'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.721026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.721371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9502d2e55c40e2b06070a081593969'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.721661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.722456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.722456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.723025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.724814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.732681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9502d2e55c40e2b06070a081593969'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.739601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 989/1957] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:09.743539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.746581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9502d2e55c40e2b06070a081593969'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.748163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:09.765254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9502d2e55c40e2b06070a081593969'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:09.768380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a4543-7593-443a-9e1d-ebb234bb51af admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.772303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 969/1958] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:09 2022] DELETE /identity/v3/users/ff9502d2e55c40e2b06070a081593969 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:09.777345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a7442-42e7-44ab-9d85-4909c6de98e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.777927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a7442-42e7-44ab-9d85-4909c6de98e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.778239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a7442-42e7-44ab-9d85-4909c6de98e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.778527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a7442-42e7-44ab-9d85-4909c6de98e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.779215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a7442-42e7-44ab-9d85-4909c6de98e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.781019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a7442-42e7-44ab-9d85-4909c6de98e0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.793470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 990/1959] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:09.798075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:09.805831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:09.906028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03b5ba-3149-44d5-8b8e-4d0b4f2444ff admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:09.912703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:09.914023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.914371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:09.914692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] PATH_INFO: `/v3/projects/8a7378ccba2f481da03ad86a0f92cfc0/users/1c4405800ad34a6bb9ad25986d0e48ba/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:09.945554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7378ccba2f481da03ad86a0f92cfc0, user_id=1c4405800ad34a6bb9ad25986d0e48ba, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:09.952191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:09.982096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 970/1960] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:09 2022] PUT /identity/v3/projects/8a7378ccba2f481da03ad86a0f92cfc0/users/1c4405800ad34a6bb9ad25986d0e48ba/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:09.993340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:09.993804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.010512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.067562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] MFA Rules not processed for user `1c4405800ad34a6bb9ad25986d0e48ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:10.069044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.139149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 991/1961] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:10.139601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:10.171748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.174042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.181779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.183480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.258717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9a4e7-cd11-4ebc-82ae-ef389cf59980 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.258987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59a26b-01af-46b6-92c8-a2ab9ca4eaad None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.263054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.263806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.263933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.264028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.264269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.264632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.264973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.265415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] PATH_INFO: `/v3/projects/0bdb2d44a069400898504c619a04a299` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.266765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdb2d44a069400898504c619a04a299)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.268963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.273782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.275272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.286472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 992/1962] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:10.312365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.323262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574013d-ccd3-4128-8e50-a97773646e0b service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.327278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdb2d44a069400898504c619a04a299', 'request_id': 'req-96a940e6-935a-4c92-b613-40f9191035de'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.333815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdb2d44a069400898504c619a04a299', 'request_id': 'req-96a940e6-935a-4c92-b613-40f9191035de'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.339737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdb2d44a069400898504c619a04a299', 'request_id': 'req-96a940e6-935a-4c92-b613-40f9191035de'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.353411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] The token cache is being invalidate because project 0bdb2d44a069400898504c619a04a299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:10.354283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.354930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-96a940e6-935a-4c92-b613-40f9191035de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.356504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 971/1963] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:10 2022] DELETE /identity/v3/projects/0bdb2d44a069400898504c619a04a299 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:10.362319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aa3f-7c3d-42a9-b288-f06a06b804f1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.362656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aa3f-7c3d-42a9-b288-f06a06b804f1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.362937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aa3f-7c3d-42a9-b288-f06a06b804f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.383454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.384546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.385055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.385578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] PATH_INFO: `/v3/users/1c4405800ad34a6bb9ad25986d0e48ba` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.387007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4405800ad34a6bb9ad25986d0e48ba)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.387451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-46b8aa3f-7c3d-42a9-b288-f06a06b804f1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:10.389770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-46b8aa3f-7c3d-42a9-b288-f06a06b804f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.390889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.402098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 972/1964] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:10.410303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.410569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.410778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.432207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:10.433417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.446624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 973/1965] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:10.450478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4405800ad34a6bb9ad25986d0e48ba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.450770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.456228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:10.459923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4405800ad34a6bb9ad25986d0e48ba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.460641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4405800ad34a6bb9ad25986d0e48ba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.461003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08beade0-0b96-4c56-b2ca-1d7d055263f9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.467121 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.467764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.468040 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.468338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.469062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:10.469389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.471014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.474281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4405800ad34a6bb9ad25986d0e48ba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.488504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 974/1966] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:10.488657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e6eeaa74-2d9b-4b49-90c4-aac85ad98dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4405800ad34a6bb9ad25986d0e48ba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:10.489461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.491176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.494113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 993/1967] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:10 2022] DELETE /identity/v3/users/1c4405800ad34a6bb9ad25986d0e48ba => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:10.498200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.498492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.498780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.506476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d39214-1b2e-4b24-b398-760e82c15cf1 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.517595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:10.518867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.532636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:10.562187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.562822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.563100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.563371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.564010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.565993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.581562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 994/1968] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:10.591565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.592942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.596692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 975/1969] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:10 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:10.597555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.654400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eac62d-d5ae-412d-8ef9-16e0960fc918 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.658908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aeff8-d33b-4987-a7c3-b7882ca4cf53 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.663623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.664246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.664528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.664801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.665431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.667387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.690369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 976/1970] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:10 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:10.694779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.703112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.703736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.704142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.704340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33fda0-1819-4d77-80af-11438183b16d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.704454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.709393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.710010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.710340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.710611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.711341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.712923 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.713875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.715495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.723704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 977/1971] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:10.725584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 995/1972] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:10.731197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.741165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5ed24-4844-4633-8b3b-efa11d85cbad admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.746438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9fe2-e5d7-4c91-b5cc-f3fddaa8c9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.747042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9fe2-e5d7-4c91-b5cc-f3fddaa8c9eb admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.747321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9fe2-e5d7-4c91-b5cc-f3fddaa8c9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.747595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9fe2-e5d7-4c91-b5cc-f3fddaa8c9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.748237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f9fe2-e5d7-4c91-b5cc-f3fddaa8c9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.749856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f9fe2-e5d7-4c91-b5cc-f3fddaa8c9eb admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.760605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 978/1973] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:10.765355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.776036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54233af-338c-4a0a-abcd-4a5c43180376 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:10.781170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:10.781892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.782187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.782459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] PATH_INFO: `/v3/projects/6bfe7873a15f4057a80008a8a5b8da3e/users/ab64e53b80c74548b253d4a1efd4ea09/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.798348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfe7873a15f4057a80008a8a5b8da3e, user_id=ab64e53b80c74548b253d4a1efd4ea09, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:10.801361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:10.815203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 996/1974] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:10 2022] PUT /identity/v3/projects/6bfe7873a15f4057a80008a8a5b8da3e/users/ab64e53b80c74548b253d4a1efd4ea09/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:10.826024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:10.826205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:10.826416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:10.855403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] MFA Rules not processed for user `ab64e53b80c74548b253d4a1efd4ea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:10.856648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:10.890129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:10.923603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 979/1975] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:10.934989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:10.936931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.001155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06fa3b-11f3-4f8e-9b56-21254fca53f6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.002292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.003744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.004753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.005386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.005721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.005930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.017558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.019241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.024355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:11.033626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 997/1976] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:11.055541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.109598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f853-5e87-40da-bf9b-35c13448b9a0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.117690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.118372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.118652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.118932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] PATH_INFO: `/v3/projects/8a7378ccba2f481da03ad86a0f92cfc0` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.120056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7378ccba2f481da03ad86a0f92cfc0)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.122023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.128028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26124af-8b26-425b-bd9f-fa5fe2443950 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.134012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.134759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.135017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.135293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] PATH_INFO: `/v3/users/ab64e53b80c74548b253d4a1efd4ea09` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.136495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab64e53b80c74548b253d4a1efd4ea09)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.138448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.179475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7378ccba2f481da03ad86a0f92cfc0', 'request_id': 'req-3c614341-1569-46fc-95c5-c7c66d4433df'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.186350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7378ccba2f481da03ad86a0f92cfc0', 'request_id': 'req-3c614341-1569-46fc-95c5-c7c66d4433df'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.191982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7378ccba2f481da03ad86a0f92cfc0', 'request_id': 'req-3c614341-1569-46fc-95c5-c7c66d4433df'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.194066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab64e53b80c74548b253d4a1efd4ea09'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.204490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab64e53b80c74548b253d4a1efd4ea09'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.205134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab64e53b80c74548b253d4a1efd4ea09'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.206175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] The token cache is being invalidate because project 8a7378ccba2f481da03ad86a0f92cfc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:11.206512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.207156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3c614341-1569-46fc-95c5-c7c66d4433df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.208942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 980/1977] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:11 2022] DELETE /identity/v3/projects/8a7378ccba2f481da03ad86a0f92cfc0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:11.214311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670eadd0-69a2-4165-9d63-e3762526b233 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.214666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670eadd0-69a2-4165-9d63-e3762526b233 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.214951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670eadd0-69a2-4165-9d63-e3762526b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.216089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab64e53b80c74548b253d4a1efd4ea09'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.232898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-91ed84ac-d1b7-418d-b901-2253e52f0e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab64e53b80c74548b253d4a1efd4ea09'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:11.236777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-670eadd0-69a2-4165-9d63-e3762526b233 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:11.238092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-670eadd0-69a2-4165-9d63-e3762526b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.239564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 998/1978] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:11 2022] DELETE /identity/v3/users/ab64e53b80c74548b253d4a1efd4ea09 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:11.243975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.244268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.244574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.263224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:11.264436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.297062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 981/1979] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:11.304719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.305013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.305266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.315488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:11.327353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 999/1980] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:11.335242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:11.336819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.337531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.339358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.345935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 982/1981] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:11.349970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.362461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:11.363690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d76acd-de38-4144-9122-c07396f8f2d7 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.370379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.371088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.371373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.371645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.372304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:11.374127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.376203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.390188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 983/1982] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:11.395354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.405874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f57243-b3cf-44b1-9391-44ea7bf9a919 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.411033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.411869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.412294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.412719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.414123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.414349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758410d-e1d7-4f24-bb5d-5e80300f5e80 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.417860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.418822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.419896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.420632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.421217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.434069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.435583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.446612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1000/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:11.455110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 984/1984] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:11.459421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.472666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad738e6-2f73-4ddc-bec1-470e2bb2d46d service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.547994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.548659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.549097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.549270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.549913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.552871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.576984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1001/1985] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:11.581640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.594568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0f932-eacb-445b-bcc8-856d0dca4a82 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.600761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.601064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.601350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.601628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.602332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.604519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.615157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 985/1986] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:11.619229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.629891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e321810-8dc5-4563-aebd-bf9a45e099e4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.634881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b82ab1-5e1b-4b59-ab83-d19c7f460282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.635505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b82ab1-5e1b-4b59-ab83-d19c7f460282 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.635795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b82ab1-5e1b-4b59-ab83-d19c7f460282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.636069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b82ab1-5e1b-4b59-ab83-d19c7f460282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.636727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b82ab1-5e1b-4b59-ab83-d19c7f460282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.638641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b82ab1-5e1b-4b59-ab83-d19c7f460282 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.650430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1002/1987] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:11.654366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.665333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff98480-a2b8-4712-b6a6-3d59099cc907 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.670554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.671233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.671499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.671784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] PATH_INFO: `/v3/projects/f2009a7bf952417691096b96f1d4700a/users/e3be0a5edd934393b98453ba8d4d0436/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.697331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2009a7bf952417691096b96f1d4700a, user_id=e3be0a5edd934393b98453ba8d4d0436, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.700003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.716911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 986/1988] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:11 2022] PUT /identity/v3/projects/f2009a7bf952417691096b96f1d4700a/users/e3be0a5edd934393b98453ba8d4d0436/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:11.731159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.731524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.731823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.760558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] MFA Rules not processed for user `e3be0a5edd934393b98453ba8d4d0436`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:11.761909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:11.829824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1003/1989] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:11.834600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.923332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0455-eb24-4c3e-b2d3-58350e22f706 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:11.929846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:11.931155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:11.931773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:11.932277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:11.933178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:11.935508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:11.974003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 987/1990] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:11.975650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:11.987080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a84515-5ec7-483a-8529-8392e68e558c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.014204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:12.047329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.048030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.048446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.048782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.049569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.051794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.074437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1004/1991] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:11 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:12.078787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.089971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b34b06-adec-4bda-892d-66feeb01ddf1 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.095806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.096547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.097006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.097364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.098092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.099748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.111360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 988/1992] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:12.115746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.127925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f382-1cdc-484a-8c5e-ba83f06be770 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.134945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.135751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.136074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.136428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.137198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.138887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.151706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1005/1993] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:12.156303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.169943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fb136-d600-4dca-a326-367f25f59a0b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.179005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81652-1732-4a02-bfe8-5d11e910874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.180408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81652-1732-4a02-bfe8-5d11e910874a admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.180700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81652-1732-4a02-bfe8-5d11e910874a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.181105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81652-1732-4a02-bfe8-5d11e910874a admin admin] PATH_INFO: `/v3/projects/e632ac434fcf4a6ab28123c486e7006a/users/6ff9a94550294158be5b9788b59d0393/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.186302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.192019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:12.201946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81652-1732-4a02-bfe8-5d11e910874a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e632ac434fcf4a6ab28123c486e7006a, user_id=6ff9a94550294158be5b9788b59d0393, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.205031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81652-1732-4a02-bfe8-5d11e910874a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.210102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbebac-9938-48e3-b693-ac4a1b63013d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.221116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.222864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.223183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.223454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] PATH_INFO: `/v3/projects/6bfe7873a15f4057a80008a8a5b8da3e` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.226259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 989/1994] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:12 2022] PUT /identity/v3/projects/e632ac434fcf4a6ab28123c486e7006a/users/6ff9a94550294158be5b9788b59d0393/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:12.226873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfe7873a15f4057a80008a8a5b8da3e)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.229503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.238681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.239023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.239260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.288731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] MFA Rules not processed for user `6ff9a94550294158be5b9788b59d0393`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:12.289971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:12.326814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfe7873a15f4057a80008a8a5b8da3e', 'request_id': 'req-7f707086-1f7d-4ef1-9c44-1914904f0810'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:12.327625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:12.340908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfe7873a15f4057a80008a8a5b8da3e', 'request_id': 'req-7f707086-1f7d-4ef1-9c44-1914904f0810'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:12.352640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfe7873a15f4057a80008a8a5b8da3e', 'request_id': 'req-7f707086-1f7d-4ef1-9c44-1914904f0810'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:12.374445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 990/1995] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:12.377782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.384876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] The token cache is being invalidate because project 6bfe7873a15f4057a80008a8a5b8da3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:12.385320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:12.386111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-7f707086-1f7d-4ef1-9c44-1914904f0810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:12.388521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1006/1996] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:12 2022] DELETE /identity/v3/projects/6bfe7873a15f4057a80008a8a5b8da3e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:12.394015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c49da-b5b6-4256-a359-cf58c64369c4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.394259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c49da-b5b6-4256-a359-cf58c64369c4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.394538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c49da-b5b6-4256-a359-cf58c64369c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.426603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2a9c49da-b5b6-4256-a359-cf58c64369c4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:12.428127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2a9c49da-b5b6-4256-a359-cf58c64369c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:12.489864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a2a16-215b-4e62-8ceb-212c47d55de0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.535226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1007/1997] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:12.542585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.542885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.543205 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.591174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:12.594405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:12.632775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1008/1998] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:12.634186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.643005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.643719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.644058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.644336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.644988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.646961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.660556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b736aff-54c4-4b70-b5c5-04010b148d86 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.672618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.680695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.681988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.681988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.681988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:12.683897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.685570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.724485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1009/1999] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:12.726970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 991/2000] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:12.730879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.732546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.741805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:12.770562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a194e-b5b9-44f8-a9ca-0bbca0eea8d6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.790092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:12.901625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3d4a3-9eef-4681-87c2-f3e22c5a0edd admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.909153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.910246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.910657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.910940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.911896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.913884 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.924763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:12.929423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.930214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.930503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.933329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.934353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.936535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:12.950730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 992/2001] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:12 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:12.954621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:12.969781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405effb-fe79-471d-9828-ceca3948325d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:12.982541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:12.986279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:12.986849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:12.987514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:12.988510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:12.992818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1010/2002] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:12.996313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.002687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.008447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.038584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 993/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:13.039518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.051927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed769a-2a90-4f1e-beaf-e2c417cb2986 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.129909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1c62-746b-471e-a340-d210a3310f30 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.134877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.135600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.135945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.136310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] PATH_INFO: `/v3/users/56844f1a822547cbbbf59e217d749240` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.137838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] RBAC: Authorizing `identity:delete_user(user_id=56844f1a822547cbbbf59e217d749240)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.139806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.145539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.146451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.146811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.147197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.148138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.150401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.177969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 994/2004] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:13 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:13.179157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.196333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a55b31-0fc9-4783-ae2a-9a556a23462f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.196600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56844f1a822547cbbbf59e217d749240'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:13.202381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.203153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.203464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.203853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] PATH_INFO: `/v3/projects/f96c17a4efe14f30826ce6345dbb6eee/users/1a031ce6ad094728824f08055ca17fbb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.207641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56844f1a822547cbbbf59e217d749240'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:13.208568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56844f1a822547cbbbf59e217d749240'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:13.224770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96c17a4efe14f30826ce6345dbb6eee, user_id=1a031ce6ad094728824f08055ca17fbb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.228560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.230340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56844f1a822547cbbbf59e217d749240'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:13.249414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 995/2005] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:13 2022] PUT /identity/v3/projects/f96c17a4efe14f30826ce6345dbb6eee/users/1a031ce6ad094728824f08055ca17fbb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:13.252058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.254318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.256653 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56844f1a822547cbbbf59e217d749240'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:13.271720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1011/2006] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:12 2022] DELETE /identity/v3/users/56844f1a822547cbbbf59e217d749240 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:13.271720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.273581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.297429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:13.339936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78386abb-06c8-465f-b656-fe054934d570 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.351289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7733d-0063-445b-ad45-dd2140ecbacd admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.356254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed8485-c045-4262-bffd-aacbfc9885e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.356883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed8485-c045-4262-bffd-aacbfc9885e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.357163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed8485-c045-4262-bffd-aacbfc9885e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.357437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed8485-c045-4262-bffd-aacbfc9885e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.358124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed8485-c045-4262-bffd-aacbfc9885e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.359765 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed8485-c045-4262-bffd-aacbfc9885e3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.369522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1012/2007] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:13.370783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.371072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.371349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.386653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:13.391126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:13.392379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.398821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.399568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.399883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.403218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1013/2008] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:13.403395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.403395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.403587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.403690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.415173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 996/2009] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:13.416967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e11c-c1e6-44b3-b76b-2f5e4f26f16e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.419213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.421127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.422854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.423533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.423812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.424085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] PATH_INFO: `/v3/projects/f96c17a4efe14f30826ce6345dbb6eee/users/1a031ce6ad094728824f08055ca17fbb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.426037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96c17a4efe14f30826ce6345dbb6eee, user_id=1a031ce6ad094728824f08055ca17fbb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.428726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.439414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1014/2010] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:13 2022] PUT /identity/v3/projects/f96c17a4efe14f30826ce6345dbb6eee/users/1a031ce6ad094728824f08055ca17fbb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:13.440329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.484118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ff3fd-709a-4781-856e-921890953ef4 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.504081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010bd62-def4-4b20-bceb-7f2fdf48863c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.508548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf85af-a0c8-44f0-9112-d2d1d9e1a15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.509162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf85af-a0c8-44f0-9112-d2d1d9e1a15a admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.509440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf85af-a0c8-44f0-9112-d2d1d9e1a15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.509708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf85af-a0c8-44f0-9112-d2d1d9e1a15a admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.510396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf85af-a0c8-44f0-9112-d2d1d9e1a15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.512025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf85af-a0c8-44f0-9112-d2d1d9e1a15a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.522125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1015/2011] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:13.523526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.523813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.524087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.540479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.541112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.541398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.541951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.552724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.553774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] MFA Rules not processed for user `1a031ce6ad094728824f08055ca17fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:13.554178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.555051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.564997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 997/2012] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:13.565953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.575991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c6e91-f9be-4c2f-8d23-b45aedc67c3e service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.581047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c18bff-b51b-4ecc-96da-5a60b3ecb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.581716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18bff-b51b-4ecc-96da-5a60b3ecb14e admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.581995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18bff-b51b-4ecc-96da-5a60b3ecb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.582557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18bff-b51b-4ecc-96da-5a60b3ecb14e admin admin] PATH_INFO: `/v3/projects/b2222f760da64228847e9991564117a7/users/8839d55aa4b44f1ca827e386a8f3de12/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.598252 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c18bff-b51b-4ecc-96da-5a60b3ecb14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2222f760da64228847e9991564117a7, user_id=8839d55aa4b44f1ca827e386a8f3de12, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.600953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c18bff-b51b-4ecc-96da-5a60b3ecb14e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.617521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 998/2013] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:13 2022] PUT /identity/v3/projects/b2222f760da64228847e9991564117a7/users/8839d55aa4b44f1ca827e386a8f3de12/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:13.628733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.629026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.629316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.636360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1016/2014] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:13.648649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.650205 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.661860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] MFA Rules not processed for user `8839d55aa4b44f1ca827e386a8f3de12`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:13.663268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.668347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:13.730862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc3368-2b4c-4302-80ed-59078d7e42f4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.750310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 999/2015] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:13.758386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:13.759926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.773715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:13.789823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.790532 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.790801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.791068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.793958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:13.824370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62cac4-de58-4807-a344-4cb1651ab013 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:13.830311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:13.831355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:13.831867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:13.832205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:13.846675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.848358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.858662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1000/2016] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:13.865899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:13.867664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:13.878369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1017/2017] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:14.138219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:14.140010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:14.254523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd1086-2fbc-42b8-baf3-d3f2db455895 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:14.261721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:14.262417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:14.262699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:14.263057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] PATH_INFO: `/v3/projects/c1e53b310c9c4b3baba0b8c8501d0481` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:14.264221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e53b310c9c4b3baba0b8c8501d0481)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:14.266195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:14.335827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e53b310c9c4b3baba0b8c8501d0481', 'request_id': 'req-28ec0148-6719-460b-93d7-500889e2f9d5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.343240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e53b310c9c4b3baba0b8c8501d0481', 'request_id': 'req-28ec0148-6719-460b-93d7-500889e2f9d5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.349568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e53b310c9c4b3baba0b8c8501d0481', 'request_id': 'req-28ec0148-6719-460b-93d7-500889e2f9d5'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.370221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] The token cache is being invalidate because project c1e53b310c9c4b3baba0b8c8501d0481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:14.370642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.373274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.375042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1001/2018] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:14 2022] DELETE /identity/v3/projects/c1e53b310c9c4b3baba0b8c8501d0481 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:14.379588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:14.406118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2fbc5-54f2-43b3-b5fa-e174b52e60b3 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:14.522216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:14.524021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:14.524630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:14.525771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] PATH_INFO: `/v3/users/ff7c98ff63044b1d9435c9ed5c1ad8e9` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:14.528303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7c98ff63044b1d9435c9ed5c1ad8e9)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:14.530943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:14.598831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:14.641858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7c98ff63044b1d9435c9ed5c1ad8e9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.658347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7c98ff63044b1d9435c9ed5c1ad8e9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.659308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7c98ff63044b1d9435c9ed5c1ad8e9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.678022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7c98ff63044b1d9435c9ed5c1ad8e9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.698541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-064bd751-3f26-4cb9-b457-8480b2f2d33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7c98ff63044b1d9435c9ed5c1ad8e9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:14.708623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1018/2019] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:14 2022] DELETE /identity/v3/users/ff7c98ff63044b1d9435c9ed5c1ad8e9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:15.026239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.028003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:15.041607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec0148-6719-460b-93d7-500889e2f9d5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.090684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:15.109266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.109916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.110224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.110498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] PATH_INFO: `/v3/projects/2b4068468a414a44ad295266d3bb21f0` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.111652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4068468a414a44ad295266d3bb21f0)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.113586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.161239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4068468a414a44ad295266d3bb21f0', 'request_id': 'req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:15.167226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4068468a414a44ad295266d3bb21f0', 'request_id': 'req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:15.172557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4068468a414a44ad295266d3bb21f0', 'request_id': 'req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:15.188000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] The token cache is being invalidate because project 2b4068468a414a44ad295266d3bb21f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:15.188315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:15.188939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-310e5bbb-010c-4adc-851f-b70b0fbbe0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:15.190473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1002/2020] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:15 2022] DELETE /identity/v3/projects/2b4068468a414a44ad295266d3bb21f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:15.196325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.196787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.197135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.220838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:15.222527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:15.289210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1019/2021] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:15.297345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.297575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.297933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.319113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:15.320347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:15.329366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1003/2022] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:15.333796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.343780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34aa6b-c1d6-4120-898b-041ebff56df9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.348436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.349051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.349334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.349603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.350313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:15.350619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.352242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.364872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1020/2023] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:41:15.369133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.379523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c25b66-bd84-4e1d-a9c5-2dda7de4249a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.385288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.388098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.388384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.388741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.389324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.391425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.430664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1004/2024] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:15 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:15.431873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.442072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5b15-b294-430f-99e0-1276bca5ef8d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.505126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.505757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.506036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.506328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.506974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.508884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.540913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1021/2025] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:15 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:15.544856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.557925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b5092-bcd5-4876-b18f-b5a65a2fd183 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.563470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.564285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.564715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.564996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.565929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.567564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.582898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1005/2026] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:15.584288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.596123 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485f68c-54a4-426b-bbd8-33caca3f9f27 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.600789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c869fd-e994-4acf-bb24-bf80fd1469f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.601405 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c869fd-e994-4acf-bb24-bf80fd1469f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.601684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c869fd-e994-4acf-bb24-bf80fd1469f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.601952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c869fd-e994-4acf-bb24-bf80fd1469f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.602632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c869fd-e994-4acf-bb24-bf80fd1469f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.604598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c869fd-e994-4acf-bb24-bf80fd1469f3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.615766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1022/2027] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:15.620498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.631324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bad0a4-6efb-4e6d-82f9-d16b36896e47 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.637005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.637917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.638299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.638802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] PATH_INFO: `/v3/projects/017a524b579e48d2bc6fe99ab8332456/users/33f584ae6eae48e994c19ae3786b95d4/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.655639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017a524b579e48d2bc6fe99ab8332456, user_id=33f584ae6eae48e994c19ae3786b95d4, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.659255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.673880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1006/2028] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:15 2022] PUT /identity/v3/projects/017a524b579e48d2bc6fe99ab8332456/users/33f584ae6eae48e994c19ae3786b95d4/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:15.685943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.686341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.686803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.712707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:15.725538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] MFA Rules not processed for user `33f584ae6eae48e994c19ae3786b95d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:15.727103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:15.809641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1023/2029] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:15.861980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:15.863849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:15.920814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975a76c-02bb-411a-8f12-97850d6954d6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:15.927420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:15.928051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:15.928503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:15.928782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:15.937982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:15.939491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:15.948816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1007/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:16.359054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:16.368803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f78aa-74e1-4750-8f98-17b8de81edea None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:16.372376 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:16.373009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:16.373284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:16.373551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:16.375301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:16.445293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:16.446803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641d5a3-06e0-486e-8643-19a9e6f43052 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:16.456958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1024/2031] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:19.107786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:19.117202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4f552-4bad-4de1-837f-902d589acf74 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:19.120951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:19.121558 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:19.121838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:19.122111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:19.123908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:19.134391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:19.190328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:19.192373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:19.207087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1008/2032] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:23.021023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a961e-4809-4102-957a-9eeff6dda380 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:23.021541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a961e-4809-4102-957a-9eeff6dda380 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:23.021541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a961e-4809-4102-957a-9eeff6dda380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:23.041380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b10a961e-4809-4102-957a-9eeff6dda380 None None] MFA Rules not processed for user `33f584ae6eae48e994c19ae3786b95d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:23.042622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b10a961e-4809-4102-957a-9eeff6dda380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:23.049995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1025/2033] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:27.193928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:27.195406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:27.263355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1c07a-f239-40cd-89d7-65ed44945c81 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:27.271859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:27.271859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:27.271859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:27.271859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] PATH_INFO: `/v3/users/8839d55aa4b44f1ca827e386a8f3de12` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:27.275702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8839d55aa4b44f1ca827e386a8f3de12)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:27.278888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:27.335454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8839d55aa4b44f1ca827e386a8f3de12'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:27.336115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8839d55aa4b44f1ca827e386a8f3de12'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:27.345366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8839d55aa4b44f1ca827e386a8f3de12'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:27.352763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8839d55aa4b44f1ca827e386a8f3de12'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:27.366389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8839d55aa4b44f1ca827e386a8f3de12'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:27.373682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1009/2034] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:27 2022] DELETE /identity/v3/users/8839d55aa4b44f1ca827e386a8f3de12 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:27.378127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:27.378671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:27.378759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:27.399851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:27.401067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:27.479334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1026/2035] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:27.485977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:27.487533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:27.523818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:27.558857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622e0c-c860-4990-9255-ef558e376658 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:27.562634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f921a5-1b6a-431a-a0fd-f17a37b4be88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:27.563255 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f921a5-1b6a-431a-a0fd-f17a37b4be88 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:27.563533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f921a5-1b6a-431a-a0fd-f17a37b4be88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:27.563813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f921a5-1b6a-431a-a0fd-f17a37b4be88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:27.573473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f921a5-1b6a-431a-a0fd-f17a37b4be88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:27.574955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f921a5-1b6a-431a-a0fd-f17a37b4be88 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:27.584911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1010/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:28.229522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.231998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.278905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e288b6-a40f-40d4-b28d-b2308880e2aa None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.283744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.284387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.284828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.285284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] PATH_INFO: `/v3/projects/b2222f760da64228847e9991564117a7` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.286500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2222f760da64228847e9991564117a7)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.288412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.330846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:28.353734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2222f760da64228847e9991564117a7', 'request_id': 'req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.367823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2222f760da64228847e9991564117a7', 'request_id': 'req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.375658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2222f760da64228847e9991564117a7', 'request_id': 'req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.386987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] The token cache is being invalidate because project b2222f760da64228847e9991564117a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:28.387649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.388351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3f661ef8-78ea-4e73-b429-06fe41a1b6c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.389983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1027/2037] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:28 2022] DELETE /identity/v3/projects/b2222f760da64228847e9991564117a7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:28.395517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.396305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.396713 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.437905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:28.439200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.521916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1011/2038] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:28.529523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.529882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.530258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.553022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:28.554385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.566668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1028/2039] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:28.572321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.582178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a5cba-fb8d-4eab-889f-10825be0b4b9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.587396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.588128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.588505 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.588913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.589859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:28.590132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.591982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.606215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1012/2040] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:41:28.610493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.621729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aea81-320c-4982-87a2-1de356bf7837 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.627273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.628221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.629342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.629543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.630223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.630677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.631670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.633852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.644434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5ce82-b9da-4c0e-a386-a65a1f9586df admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.653649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.653649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.653649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.653649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] PATH_INFO: `/v3/users/e3be0a5edd934393b98453ba8d4d0436` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.654119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3be0a5edd934393b98453ba8d4d0436)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.656063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.684956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1029/2041] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:28 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:28.686005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.728592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3be0a5edd934393b98453ba8d4d0436'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.729621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3be0a5edd934393b98453ba8d4d0436'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.739086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3be0a5edd934393b98453ba8d4d0436'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.747239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3be0a5edd934393b98453ba8d4d0436'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.759674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1262d2bb-6240-4d1c-b494-a846eda9d350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3be0a5edd934393b98453ba8d4d0436'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:28.770188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1013/2042] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:28 2022] DELETE /identity/v3/users/e3be0a5edd934393b98453ba8d4d0436 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:28.772663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.772794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e6f29-7dc3-414c-ab66-7d080f544313 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.772951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.773422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.792707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:28.793965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.817624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:28.827786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.828324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.828589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.828863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.829488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.831481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.854659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1030/2043] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:28 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:28.857912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.859190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1014/2044] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:28.859580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.868980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.870345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:28.871814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d7229-b4e0-4b1f-ba7b-ed3e9d29ff51 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.876666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.877348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.877625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.877893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.878960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.880287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.891082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1031/2045] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:28.895141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.906889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0f21-f318-4553-bbe9-a5f7406a4422 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.912656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.913315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.913595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.913867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.914538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.916244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.921729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:28.928834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c29b-d250-42f6-a3a9-9d2a5288c710 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.931852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1032/2046] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:28.932703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33a401-2f23-4f3a-90ee-61bcd15dd6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.932833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:28.933348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33a401-2f23-4f3a-90ee-61bcd15dd6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.933632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33a401-2f23-4f3a-90ee-61bcd15dd6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.933904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33a401-2f23-4f3a-90ee-61bcd15dd6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.943694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33a401-2f23-4f3a-90ee-61bcd15dd6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.945025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31b34f-35a8-40a0-a111-8d9a6dfc7a50 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:28.945138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33a401-2f23-4f3a-90ee-61bcd15dd6f2 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.950480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:28.951174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:28.951538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:28.951768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] PATH_INFO: `/v3/projects/9cd21f96d5df4ba2afe15aa143648788/users/a0f110e337974435bd065c5995234fe3/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:28.955028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1015/2047] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:28.969251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd21f96d5df4ba2afe15aa143648788, user_id=a0f110e337974435bd065c5995234fe3, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:28.972611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:28.987763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1033/2048] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:28 2022] PUT /identity/v3/projects/9cd21f96d5df4ba2afe15aa143648788/users/a0f110e337974435bd065c5995234fe3/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:28.999819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:29.000092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:29.000411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:29.034553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] MFA Rules not processed for user `a0f110e337974435bd065c5995234fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:29.036059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:29.117061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1016/2049] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:29.125619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:29.127931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:29.209164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb47447-eefb-494b-b73e-60fae779b7fd admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:29.214742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:29.215538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:29.216192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:29.216598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:29.230631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:29.232153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:29.247021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1034/2050] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:29.337748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:29.340445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:29.484193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454c66-da5e-47dc-86a3-37a3cc4abdf5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:29.488980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:29.490580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:29.491023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:29.491297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] PATH_INFO: `/v3/projects/f2009a7bf952417691096b96f1d4700a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:29.492452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2009a7bf952417691096b96f1d4700a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:29.494462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:29.497707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:29.544818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2009a7bf952417691096b96f1d4700a', 'request_id': 'req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.552451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2009a7bf952417691096b96f1d4700a', 'request_id': 'req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.558734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2009a7bf952417691096b96f1d4700a', 'request_id': 'req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.575233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] The token cache is being invalidate because project f2009a7bf952417691096b96f1d4700a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:29.575587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.576284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.579630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1017/2051] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:29 2022] DELETE /identity/v3/projects/f2009a7bf952417691096b96f1d4700a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:29.582451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:29.595573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b95c-923e-49c2-8d62-b30f649218c8 service glance] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:29.669241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:29.669988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:29.670313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:29.670704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] PATH_INFO: `/v3/users/6ff9a94550294158be5b9788b59d0393` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:29.671998 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff9a94550294158be5b9788b59d0393)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:29.674132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:29.726269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff9a94550294158be5b9788b59d0393'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.734923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff9a94550294158be5b9788b59d0393'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.735626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff9a94550294158be5b9788b59d0393'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.745067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff9a94550294158be5b9788b59d0393'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.763753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff9a94550294158be5b9788b59d0393'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:29.767100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:29.774210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1035/2052] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:29 2022] DELETE /identity/v3/users/6ff9a94550294158be5b9788b59d0393 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:30.089934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:30.091647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:30.108768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1f98b-e520-40b1-bb2e-3b7abe6b8d27 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:30.170411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:30.171051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:30.171331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:30.171599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] PATH_INFO: `/v3/projects/e632ac434fcf4a6ab28123c486e7006a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:30.172716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e632ac434fcf4a6ab28123c486e7006a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:30.174653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:30.223941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e632ac434fcf4a6ab28123c486e7006a', 'request_id': 'req-4ce43ea3-ba0b-45db-87fb-22f037540698'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.231141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e632ac434fcf4a6ab28123c486e7006a', 'request_id': 'req-4ce43ea3-ba0b-45db-87fb-22f037540698'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.237362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e632ac434fcf4a6ab28123c486e7006a', 'request_id': 'req-4ce43ea3-ba0b-45db-87fb-22f037540698'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.251974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] The token cache is being invalidate because project e632ac434fcf4a6ab28123c486e7006a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:30.252262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.252873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.254430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1018/2053] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:30 2022] DELETE /identity/v3/projects/e632ac434fcf4a6ab28123c486e7006a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:30.258392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:30.260042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:30.275889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af9cfb-80e4-41d2-8e7c-27405bf90ae2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:30.341312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:30.341970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:30.342389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:30.342676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] PATH_INFO: `/v3/users/1a031ce6ad094728824f08055ca17fbb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:30.343830 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a031ce6ad094728824f08055ca17fbb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:30.345768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:30.401738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a031ce6ad094728824f08055ca17fbb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.425748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a031ce6ad094728824f08055ca17fbb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.426432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a031ce6ad094728824f08055ca17fbb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.435913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a031ce6ad094728824f08055ca17fbb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.452741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a031ce6ad094728824f08055ca17fbb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.462911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1036/2054] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:30 2022] DELETE /identity/v3/users/1a031ce6ad094728824f08055ca17fbb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:30.820324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:30.821995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:30.837856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce43ea3-ba0b-45db-87fb-22f037540698 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:30.867463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:30.927607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:30.928274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:30.928707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:30.928978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] PATH_INFO: `/v3/projects/f96c17a4efe14f30826ce6345dbb6eee` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:30.930376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96c17a4efe14f30826ce6345dbb6eee)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:30.932441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:30.953326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:30.954848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:30.968494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d761af-34f3-4e8a-b7cd-ee1d45b91243 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:30.988733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96c17a4efe14f30826ce6345dbb6eee', 'request_id': 'req-93e24353-bfcd-405f-84bf-7ad92d75dfd8'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:30.995132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96c17a4efe14f30826ce6345dbb6eee', 'request_id': 'req-93e24353-bfcd-405f-84bf-7ad92d75dfd8'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.001478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96c17a4efe14f30826ce6345dbb6eee', 'request_id': 'req-93e24353-bfcd-405f-84bf-7ad92d75dfd8'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.017465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] The token cache is being invalidate because project f96c17a4efe14f30826ce6345dbb6eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:31.017814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.018995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-93e24353-bfcd-405f-84bf-7ad92d75dfd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.021237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1019/2055] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:30 2022] DELETE /identity/v3/projects/f96c17a4efe14f30826ce6345dbb6eee => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:31.026750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92cd17-0d9b-4c1a-adcc-1f596f6330aa None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.027227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92cd17-0d9b-4c1a-adcc-1f596f6330aa None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.027668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92cd17-0d9b-4c1a-adcc-1f596f6330aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.032096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.032751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.033030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.033302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] PATH_INFO: `/v3/users/0e155fba6fe445cfacc865480348c3ae` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.034469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e155fba6fe445cfacc865480348c3ae)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.036461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.056318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-ae92cd17-0d9b-4c1a-adcc-1f596f6330aa None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:31.057817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ae92cd17-0d9b-4c1a-adcc-1f596f6330aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.061783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:31.082420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1020/2056] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:31.087647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.087939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.088207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.096460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e155fba6fe445cfacc865480348c3ae'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.106867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e155fba6fe445cfacc865480348c3ae'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.108240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e155fba6fe445cfacc865480348c3ae'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.110194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:31.111514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.118026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e155fba6fe445cfacc865480348c3ae'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.119732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1021/2057] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:31.124374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.132679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a317b921-602c-4345-84eb-6f67d87a51c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e155fba6fe445cfacc865480348c3ae'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:31.135780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b57c10-bcab-4ed2-a613-b03f73e91898 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.140490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1037/2058] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:30 2022] DELETE /identity/v3/users/0e155fba6fe445cfacc865480348c3ae => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:31.143553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.143553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.143553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.143553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.143952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:31.143952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.144593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.145195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.145546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.145886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.157905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1022/2059] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:31.162008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.168643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:31.169979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.172542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deea6f0-4937-4f84-80f0-8277a1e50890 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.254671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.256631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.257246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.257655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.259636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.260119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1038/2060] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:31.261798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.267547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.269129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.298642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1023/2061] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:31 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:31.302371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.321366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:31.390365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc2bc-4748-4d46-9877-e6b240ffc97c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.435115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8629b-5de4-4eb9-b64f-e7fe33184cf4 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.446649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.447289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.447568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.447836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.448467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.453560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.480354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.481341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.481619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.481978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.490440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:31.493690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.498224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.498379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1024/2062] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:31 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:31.499714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.511947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1039/2063] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:41:31.538513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c1103-ce5c-4ee6-a73f-162828010930 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.544681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.545406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.545762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.546109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.546878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.548756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.565829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1025/2064] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:31.569408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.580858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f676a05-3c1e-4212-89f2-6070b2adf503 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.588995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b15c3-88f6-4da4-adfd-5c2c7de42fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.590038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b15c3-88f6-4da4-adfd-5c2c7de42fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.591071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b15c3-88f6-4da4-adfd-5c2c7de42fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.591726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b15c3-88f6-4da4-adfd-5c2c7de42fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.592576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b15c3-88f6-4da4-adfd-5c2c7de42fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.594707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b15c3-88f6-4da4-adfd-5c2c7de42fa7 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.610196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1040/2065] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:31.613727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.627866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e61678-0df9-42d4-85c6-a755e8874970 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.636277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.637033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.637889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.637889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] PATH_INFO: `/v3/projects/f0a5508bdf5e4577b49783c3e424f1ee/users/f6d6b64eb434414ab61f19b170385c3d/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.658598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a5508bdf5e4577b49783c3e424f1ee, user_id=f6d6b64eb434414ab61f19b170385c3d, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.661741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.685686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1026/2066] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:31 2022] PUT /identity/v3/projects/f0a5508bdf5e4577b49783c3e424f1ee/users/f6d6b64eb434414ab61f19b170385c3d/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:31.697236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.697649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.698033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.723789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:31.725143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.815476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1041/2067] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:31.831856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.834270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.849317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:31.855514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:31.870973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b60668-086b-4533-b45f-521bb6e75041 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:31.877766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:31.878757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:31.879241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:31.879721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] PATH_INFO: `/v3/projects/d777960bc55f4471abd193a7df4a6200` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:31.881371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d777960bc55f4471abd193a7df4a6200)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:31.883793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:31.996078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd777960bc55f4471abd193a7df4a6200', 'request_id': 'req-1dc36120-4276-44e1-a976-450d383272f8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.009906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d00b2-4d84-4c51-86cc-7dacea4c20b7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:32.028656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd777960bc55f4471abd193a7df4a6200', 'request_id': 'req-1dc36120-4276-44e1-a976-450d383272f8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.042202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd777960bc55f4471abd193a7df4a6200', 'request_id': 'req-1dc36120-4276-44e1-a976-450d383272f8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.063789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] The token cache is being invalidate because project d777960bc55f4471abd193a7df4a6200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:32.064083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.064783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.067000 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1042/2068] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:31 2022] DELETE /identity/v3/projects/d777960bc55f4471abd193a7df4a6200 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:32.072171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:32.073697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:32.111738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:32.112370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:32.112651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:32.113757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:32.115702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:32.162649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:32.177678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc36120-4276-44e1-a976-450d383272f8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:32.182664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:32.183347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:32.183643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:32.184025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] PATH_INFO: `/v3/users/006e8964a50a417d8d5cda9d8bc422a1` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:32.185083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006e8964a50a417d8d5cda9d8bc422a1)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:32.187081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:32.194487 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:32.196019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:32.199614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:32.206440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1027/2069] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:32.246737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006e8964a50a417d8d5cda9d8bc422a1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.267852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006e8964a50a417d8d5cda9d8bc422a1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.268646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006e8964a50a417d8d5cda9d8bc422a1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.278111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006e8964a50a417d8d5cda9d8bc422a1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.294078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1255f89c-7654-4741-bccb-422788c379b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006e8964a50a417d8d5cda9d8bc422a1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:32.305463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1043/2070] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:32 2022] DELETE /identity/v3/users/006e8964a50a417d8d5cda9d8bc422a1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:32.892862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:32.903820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6edf-df01-4950-9ce1-ec048b758636 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.025593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.026437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.026790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.027301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] PATH_INFO: `/v3/projects/58b6ff2e844e4041a64946b89fe5ff29` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.028641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b6ff2e844e4041a64946b89fe5ff29)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.030814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.142600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b6ff2e844e4041a64946b89fe5ff29', 'request_id': 'req-59f87e40-2831-4d8f-b65f-f61454dcda1d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:33.151216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b6ff2e844e4041a64946b89fe5ff29', 'request_id': 'req-59f87e40-2831-4d8f-b65f-f61454dcda1d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:33.158621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b6ff2e844e4041a64946b89fe5ff29', 'request_id': 'req-59f87e40-2831-4d8f-b65f-f61454dcda1d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:33.174258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] The token cache is being invalidate because project 58b6ff2e844e4041a64946b89fe5ff29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:33.174863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:33.175782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-59f87e40-2831-4d8f-b65f-f61454dcda1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:33.178192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1028/2071] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:32 2022] DELETE /identity/v3/projects/58b6ff2e844e4041a64946b89fe5ff29 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:33.183825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.183825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.184432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.207658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:33.209055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:33.281068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1044/2072] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:33.290382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.291108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.292329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.320540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:33.322572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:33.334892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1029/2073] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:33.339889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.355704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f81a8-e58b-409c-aa4e-a1f18d6e7a57 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.362955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.363625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.364001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.364260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.364925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:33.365226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.366854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.380595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1045/2074] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:41:33.385145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.397047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e13a1-2041-405a-a870-f1aefebc6dbf None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.401859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.402515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.402802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.403054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.403688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.405873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.440030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1030/2075] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:33 2022] POST /identity/v3/projects => generated 407 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:33.444594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.456202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a1fbb-ecdf-42ea-91f1-ca89b03fbccc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.518513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.518802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.519091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.519731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.519986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.522047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.544204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1046/2076] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:33 2022] POST /identity/v3/users => generated 304 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:33.548384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.558716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae167ec-f33b-4b39-9f78-0a1bcb50de32 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.563299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.563903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.564179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.564448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.565089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.566728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.577092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1031/2077] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:41:33.581165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.588002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:33.592810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134bf10-6863-47d1-a416-3ffd2518da32 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.600283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7596ae-4ab5-440a-9e5a-582e2f752cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.600920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7596ae-4ab5-440a-9e5a-582e2f752cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.601209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7596ae-4ab5-440a-9e5a-582e2f752cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.601485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7596ae-4ab5-440a-9e5a-582e2f752cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.602268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7596ae-4ab5-440a-9e5a-582e2f752cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.604481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7596ae-4ab5-440a-9e5a-582e2f752cf3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.620803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1047/2078] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:33.624620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.637918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84877b8-9542-403d-8ee7-6559ac945a33 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.645925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.646656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.646937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.647223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] PATH_INFO: `/v3/projects/7e98a69eeea24c399fc696a0145f6245/users/7603d4a8f92540f4b9697699be0111b4/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.673652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e98a69eeea24c399fc696a0145f6245, user_id=7603d4a8f92540f4b9697699be0111b4, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.676376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:33.689311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:33.702432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1032/2079] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:33 2022] PUT /identity/v3/projects/7e98a69eeea24c399fc696a0145f6245/users/7603d4a8f92540f4b9697699be0111b4/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:33.707529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.707826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.708107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.755648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] MFA Rules not processed for user `7603d4a8f92540f4b9697699be0111b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:33.757015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:33.833976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1048/2080] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:33 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:33.838921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:33.941351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9553fd-7ce0-408b-b773-f1ce67ce95dd admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:33.953347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:33.954164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:33.955026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:33.955026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:33.956757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:33.958925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.002204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1033/2081] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:33 2022] POST /identity/v3/projects => generated 409 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:34.004265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:34.016381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1104c-b686-4d56-bcd3-67dc916e281c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:34.101278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:34.103728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.103728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.103728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.104417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:34.114853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.151061 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1049/2082] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:34 2022] POST /identity/v3/users => generated 311 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:34.154960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:34.175373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc11f-c3fc-4751-ae36-a9b24b7e33a3 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:34.188108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:34.188899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.189194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.189469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.190139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:34.192160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.215662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1034/2083] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:34.219756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:34.239754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b3cca-bcc1-4e08-a153-e79e5956de4d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:34.249855 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:34.251023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.251023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.251227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] PATH_INFO: `/v3/projects/7f4da68344a644aca8fd7960f0a55d60/users/8fdab6befdb74e2eb378bfb6c674dfa1/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.312562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4da68344a644aca8fd7960f0a55d60, user_id=8fdab6befdb74e2eb378bfb6c674dfa1, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:34.315308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.338288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1050/2084] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:34 2022] PUT /identity/v3/projects/7f4da68344a644aca8fd7960f0a55d60/users/8fdab6befdb74e2eb378bfb6c674dfa1/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:34.343932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:34.355962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa1123-d42c-43c5-b25b-3d0596599f3b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:34.513319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059749b7-8ed9-4c1c-9266-e02d3778ec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:34.513943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059749b7-8ed9-4c1c-9266-e02d3778ec99 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.515449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059749b7-8ed9-4c1c-9266-e02d3778ec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.515860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059749b7-8ed9-4c1c-9266-e02d3778ec99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.516571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059749b7-8ed9-4c1c-9266-e02d3778ec99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:34.518270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059749b7-8ed9-4c1c-9266-e02d3778ec99 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.550383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1035/2085] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:34.554520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:34.567956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81912-53b1-4f56-b33c-361014614dd5 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:34.581793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:34.582529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.582811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.583084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] PATH_INFO: `/v3/projects/7f4da68344a644aca8fd7960f0a55d60/users/8fdab6befdb74e2eb378bfb6c674dfa1/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.591404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4da68344a644aca8fd7960f0a55d60, user_id=8fdab6befdb74e2eb378bfb6c674dfa1, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:34.598757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.635421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1051/2086] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:34 2022] PUT /identity/v3/projects/7f4da68344a644aca8fd7960f0a55d60/users/8fdab6befdb74e2eb378bfb6c674dfa1/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:34.645953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.646299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.646553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.700416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] MFA Rules not processed for user `8fdab6befdb74e2eb378bfb6c674dfa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:34.702327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:34.798299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:34.814892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1036/2087] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:34 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:34.826978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:34.828762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:34.902232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:34.923400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b778-c044-495a-8c78-bf52f681e31b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:34.927985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba08160-6c79-4eaf-8aa0-2e45e25b95c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:34.928659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba08160-6c79-4eaf-8aa0-2e45e25b95c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:34.928944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba08160-6c79-4eaf-8aa0-2e45e25b95c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:34.929223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba08160-6c79-4eaf-8aa0-2e45e25b95c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:34.940108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba08160-6c79-4eaf-8aa0-2e45e25b95c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:34.942518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba08160-6c79-4eaf-8aa0-2e45e25b95c1 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:34.954694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1052/2088] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:34 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:35.184971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:35.198543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73693b2-08dc-4bc7-b574-83edd08fcab3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:35.203441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:35.204392 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:35.205152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:35.205288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:35.311718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:35.313361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:35.324282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1037/2089] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:35 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:38.423958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.424448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:38.424647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:38.456593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] MFA Rules not processed for user `f6d6b64eb434414ab61f19b170385c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:38.458297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:38.544985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1053/2090] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:38.548967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:38.550570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:38.630858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3679382-ae3d-4bfd-9987-0226c3b07683 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:38.635778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:38.636421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.636703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:38.636977 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:38.637922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:38.640005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:38.677606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1038/2091] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:38 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:41:38.681797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:38.692744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7542410-ec8c-4b98-b410-60d9738564b8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:38.760934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:38.761564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.761847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:38.762121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:38.763177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:38.765280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:38.789656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1054/2092] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:38 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:38.793992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:38.805811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c36b8-c6b7-430d-931d-bae19873e01c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:38.810992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:38.811790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.812079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:38.812354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:38.813023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:38.814702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:38.831787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1039/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:38.833702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:38.845625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0bd64-e292-42ac-b99e-66138ef1b087 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:38.850686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:38.851384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.851667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:38.851948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] PATH_INFO: `/v3/projects/5099eb118dd4449a9dea3ece36073723/users/22f889d9fd8f4ce9883ec29e5951fb67/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:38.869536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5099eb118dd4449a9dea3ece36073723, user_id=22f889d9fd8f4ce9883ec29e5951fb67, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:38.872292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:38.887649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1055/2094] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:38 2022] PUT /identity/v3/projects/5099eb118dd4449a9dea3ece36073723/users/22f889d9fd8f4ce9883ec29e5951fb67/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:41:38.892023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:38.902599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0b9d-1784-4cd8-bb69-afcbf1cb00ff admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:38.962398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:38.963305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.963575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:38.963953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:38.964718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:38.966693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:38.977798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1040/2095] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:38.982625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:38.993600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0dc5c-2a57-4624-8c73-a51de27fda37 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:38.998984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf480e0-1c85-4d60-aaf5-db4512b97946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:38.999639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf480e0-1c85-4d60-aaf5-db4512b97946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:38.999928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf480e0-1c85-4d60-aaf5-db4512b97946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:39.000201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf480e0-1c85-4d60-aaf5-db4512b97946 admin admin] PATH_INFO: `/v3/projects/5099eb118dd4449a9dea3ece36073723/users/22f889d9fd8f4ce9883ec29e5951fb67/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:39.002262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf480e0-1c85-4d60-aaf5-db4512b97946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5099eb118dd4449a9dea3ece36073723, user_id=22f889d9fd8f4ce9883ec29e5951fb67, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:39.004927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf480e0-1c85-4d60-aaf5-db4512b97946 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:39.016045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1056/2096] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:38 2022] PUT /identity/v3/projects/5099eb118dd4449a9dea3ece36073723/users/22f889d9fd8f4ce9883ec29e5951fb67/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:44.771143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:44.777090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:44.787287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aeebfe-c4f5-469a-a892-0dbbf2dfc26c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:44.932836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:44.934107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:44.934361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:44.934640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] PATH_INFO: `/v3/users/7603d4a8f92540f4b9697699be0111b4` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:44.936011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7603d4a8f92540f4b9697699be0111b4)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:44.937873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:45.060482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7603d4a8f92540f4b9697699be0111b4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:45.062381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7603d4a8f92540f4b9697699be0111b4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:45.087915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7603d4a8f92540f4b9697699be0111b4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:45.099731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7603d4a8f92540f4b9697699be0111b4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:45.125411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7603d4a8f92540f4b9697699be0111b4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:45.151525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1041/2097] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:44 2022] DELETE /identity/v3/users/7603d4a8f92540f4b9697699be0111b4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:45.152665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b41f8-f254-49e2-8acf-1c5815213533 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:45.153225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b41f8-f254-49e2-8acf-1c5815213533 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:45.153395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b41f8-f254-49e2-8acf-1c5815213533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:45.186077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6b1b41f8-f254-49e2-8acf-1c5815213533 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:45.188143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6b1b41f8-f254-49e2-8acf-1c5815213533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:45.214103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:45.334251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1057/2098] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:45.344845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:45.346597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:45.437231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da6f77-6881-4e8b-87c8-26e634799266 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:45.441058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:45.441716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:45.441987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:45.442300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:45.461182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:45.463088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:45.477374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1042/2099] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:45.546906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:45.547704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:45.547704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:45.614800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] MFA Rules not processed for user `22f889d9fd8f4ce9883ec29e5951fb67`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:45.616326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:45.720570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1058/2100] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:45.740450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:45.742465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:45.782896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:45.834439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d2bd9-6ae3-46c0-8a48-61858d2372f8 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:45.840857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:45.841670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:45.842011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:45.843061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:45.847817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:45.964177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:45.965578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab533a8-97c3-4cb2-9e25-1c693c7c4ec9 service glance] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:45.989673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1043/2101] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:46.084449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:46.086083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:46.108745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cbd1e-5166-4330-98b5-7f44ccc1712f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:46.121133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:46.122567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:46.122951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:46.123337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] PATH_INFO: `/v3/projects/7e98a69eeea24c399fc696a0145f6245` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:46.125608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e98a69eeea24c399fc696a0145f6245)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:46.128135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:46.216768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e98a69eeea24c399fc696a0145f6245', 'request_id': 'req-956eb24f-cc83-4a3b-8e4e-991b02fe4504'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.228018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e98a69eeea24c399fc696a0145f6245', 'request_id': 'req-956eb24f-cc83-4a3b-8e4e-991b02fe4504'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.234420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:46.235986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:46.236452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:46.244047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e98a69eeea24c399fc696a0145f6245', 'request_id': 'req-956eb24f-cc83-4a3b-8e4e-991b02fe4504'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.259498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] The token cache is being invalidate because project 7e98a69eeea24c399fc696a0145f6245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:46.259807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.260604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.265218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1059/2102] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:46 2022] DELETE /identity/v3/projects/7e98a69eeea24c399fc696a0145f6245 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:46.266543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:46.269326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:46.271119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:46.272766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:46.339701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:46.383127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1044/2103] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:41:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:46.402288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb24f-cc83-4a3b-8e4e-991b02fe4504 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:46.431471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:46.432159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:46.432434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:46.432702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] PATH_INFO: `/v3/users/8fdab6befdb74e2eb378bfb6c674dfa1` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:46.434687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:46.436247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:46.442438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdab6befdb74e2eb378bfb6c674dfa1)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:46.446459 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:46.507178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdab6befdb74e2eb378bfb6c674dfa1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.517607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdab6befdb74e2eb378bfb6c674dfa1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.518600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdab6befdb74e2eb378bfb6c674dfa1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.528817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdab6befdb74e2eb378bfb6c674dfa1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.533064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d55f1d-2c86-4131-8840-aa24fc004c39 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:46.540427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:46.541827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:46.542202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:46.542406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdab6befdb74e2eb378bfb6c674dfa1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:46.542663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:46.545390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:46.549670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1060/2104] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:46 2022] DELETE /identity/v3/users/8fdab6befdb74e2eb378bfb6c674dfa1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:46.567110 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:46.568942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b2370-a95b-4e48-b43d-2055acb218bc service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:46.578017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1045/2105] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:41:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:41:47.038239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:47.040141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:47.252102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4e0b8-bee3-4e21-823d-b1d1d186be8b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:47.270085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:47.271175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.271621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.272054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] PATH_INFO: `/v3/projects/7f4da68344a644aca8fd7960f0a55d60` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.273246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4da68344a644aca8fd7960f0a55d60)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:47.275280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:47.351542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4da68344a644aca8fd7960f0a55d60', 'request_id': 'req-b5f94e24-28bb-447a-872a-1089fa099cfd'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:47.353775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:47.377899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4da68344a644aca8fd7960f0a55d60', 'request_id': 'req-b5f94e24-28bb-447a-872a-1089fa099cfd'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:47.396922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4da68344a644aca8fd7960f0a55d60', 'request_id': 'req-b5f94e24-28bb-447a-872a-1089fa099cfd'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:47.417984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] The token cache is being invalidate because project 7f4da68344a644aca8fd7960f0a55d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:47.418587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:47.420187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b5f94e24-28bb-447a-872a-1089fa099cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:47.422228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1061/2106] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:47 2022] DELETE /identity/v3/projects/7f4da68344a644aca8fd7960f0a55d60 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:47.427319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.427827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.427919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.449168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:47.450435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:47.461016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:47.531312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1046/2107] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:47.542487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.543252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.543686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.576408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:47.577965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:47.594223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1062/2108] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:47.598792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:47.612544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1e9bf-51d3-486b-947d-c0d65b1a8aae None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:47.627004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:47.628574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.629103 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.629826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.630686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:47.631488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:47.633221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:47.653404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1047/2109] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:47.658049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:47.669703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcddda-f363-4514-974d-6884ab940151 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:47.679812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:47.680964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.681465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.682014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.683392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:47.685917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:47.735221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1063/2110] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:47 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:47.740527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:47.771458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab68d7-1f3f-41c2-b770-2e3f55552b07 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:47.868383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:47.869367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.869854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.870393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.872309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:47.875823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:47.919433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1048/2111] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:47 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:47.924436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:47.935848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4072-72ec-45be-81ad-2477aa1435ea admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:47.948086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:47.948736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.949022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.949294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:47.949962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:47.951630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:47.964699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1064/2112] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:47.969163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:47.987359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07548e-70b8-488f-abb6-ffb3af2d808d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:47.993329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abb253-9c88-47ea-b36f-77cbcb6777ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:47.998733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abb253-9c88-47ea-b36f-77cbcb6777ca admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:47.999163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abb253-9c88-47ea-b36f-77cbcb6777ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:47.999714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abb253-9c88-47ea-b36f-77cbcb6777ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:48.000617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abb253-9c88-47ea-b36f-77cbcb6777ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:48.002402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abb253-9c88-47ea-b36f-77cbcb6777ca admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:48.022801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1049/2113] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:48.027158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:48.038066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33eed9-20aa-462c-a42e-ca05220d9660 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:48.043212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:48.043904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:48.044180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:48.044447 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] PATH_INFO: `/v3/projects/eb576abacf4a41b59056aeb0b683225c/users/e0165ad1bd7147e1a9dded684004208f/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:48.061079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb576abacf4a41b59056aeb0b683225c, user_id=e0165ad1bd7147e1a9dded684004208f, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:48.063969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:48.083245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1065/2114] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:48 2022] PUT /identity/v3/projects/eb576abacf4a41b59056aeb0b683225c/users/e0165ad1bd7147e1a9dded684004208f/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:48.088427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba51fd-3efd-44b7-91a8-945c243bfea1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:48.089106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba51fd-3efd-44b7-91a8-945c243bfea1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:48.089541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba51fd-3efd-44b7-91a8-945c243bfea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:48.115377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-83ba51fd-3efd-44b7-91a8-945c243bfea1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:48.116638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-83ba51fd-3efd-44b7-91a8-945c243bfea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:48.185937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1050/2115] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:48.196074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:48.198444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:48.335288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7cc37-db85-4b25-9f2c-844307db3977 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:48.365417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:48.366374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:48.366817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:48.367245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:48.391334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:48.393349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:48.426104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1066/2116] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:50.612634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:50.613261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:50.613261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:50.689794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] MFA Rules not processed for user `e0165ad1bd7147e1a9dded684004208f`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:50.692701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:50.715625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:50.852995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:50.865871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0b888-8cf4-4237-bca4-ab51b460756c service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:50.866644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1051/2117] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:50.872141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:50.873252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:50.873759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:50.874294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:50.875175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:50.877344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:50.926576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:50.927482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1067/2118] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:50 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:50.942857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1753-4afd-4232-9687-59160e17ce9d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:51.119878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:51.120623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:51.121025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:51.121400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:51.122142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:51.127701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:51.195766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1052/2119] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:50 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:51.200832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:51.231594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9742e-50ff-4bc8-b6ff-24281dc2d61f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:51.237603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:51.239472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:51.239933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:51.240613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:51.241407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:51.244203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:51.263086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1068/2120] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:51.267781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:51.286881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dafd1-bdd8-4505-8283-de1343439b91 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:51.298515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:51.299067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:51.306932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:51.307429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] PATH_INFO: `/v3/projects/97312962054e4963bc241a74a8142752/users/c48a57cfd87f43b992d1ad55c24898e4/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:51.347827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97312962054e4963bc241a74a8142752, user_id=c48a57cfd87f43b992d1ad55c24898e4, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:51.350663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:51.378025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1053/2121] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:51 2022] PUT /identity/v3/projects/97312962054e4963bc241a74a8142752/users/c48a57cfd87f43b992d1ad55c24898e4/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:51.385107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:51.399504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93e93-c5d7-4c63-991a-0b094f2987ba admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:51.465283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:51.490212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d4393-54f8-4c6b-8918-92deef0ec1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:51.491529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4393-54f8-4c6b-8918-92deef0ec1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:51.492036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4393-54f8-4c6b-8918-92deef0ec1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:51.492411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4393-54f8-4c6b-8918-92deef0ec1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:51.493216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d4393-54f8-4c6b-8918-92deef0ec1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:51.494839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d4393-54f8-4c6b-8918-92deef0ec1cd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:51.524252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1069/2122] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:51.525073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:51.537637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce41a-7ad9-470a-a11d-b43fee474313 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:51.545341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:51.546038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:51.546346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:51.546617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] PATH_INFO: `/v3/projects/97312962054e4963bc241a74a8142752/users/c48a57cfd87f43b992d1ad55c24898e4/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:51.550389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97312962054e4963bc241a74a8142752, user_id=c48a57cfd87f43b992d1ad55c24898e4, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:51.553190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:51.572492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1054/2123] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:51 2022] PUT /identity/v3/projects/97312962054e4963bc241a74a8142752/users/c48a57cfd87f43b992d1ad55c24898e4/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:52.787289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c5c7-9002-4b46-a9aa-abb3ca2d66a7 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:52.788494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c5c7-9002-4b46-a9aa-abb3ca2d66a7 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:52.788965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c5c7-9002-4b46-a9aa-abb3ca2d66a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:52.841677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6916c5c7-9002-4b46-a9aa-abb3ca2d66a7 None None] MFA Rules not processed for user `c48a57cfd87f43b992d1ad55c24898e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:52.843105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6916c5c7-9002-4b46-a9aa-abb3ca2d66a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:52.962539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1070/2124] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:52.972673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:52.974696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:53.148054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee261d3-250d-4e7a-8c58-90d9a5de0f90 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:53.158555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a655d-d7d0-4451-b44b-757a11f9eaf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:53.159986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a655d-d7d0-4451-b44b-757a11f9eaf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:53.160421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a655d-d7d0-4451-b44b-757a11f9eaf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:53.160982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a655d-d7d0-4451-b44b-757a11f9eaf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:53.194585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:53.286636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a655d-d7d0-4451-b44b-757a11f9eaf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:53.290655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a655d-d7d0-4451-b44b-757a11f9eaf7 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:53.324811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1055/2125] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:54.730307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc039a-46c3-4283-8de3-8588ec44777c None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:54.730879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc039a-46c3-4283-8de3-8588ec44777c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:54.730879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc039a-46c3-4283-8de3-8588ec44777c None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:54.732243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1071/2126] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:41:54.744786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918bec-eb0a-48f9-af97-054517f236ff None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:54.745210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918bec-eb0a-48f9-af97-054517f236ff None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:54.745416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918bec-eb0a-48f9-af97-054517f236ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:54.787379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e5918bec-eb0a-48f9-af97-054517f236ff None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:54.794109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e5918bec-eb0a-48f9-af97-054517f236ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:54.825959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc039a-46c3-4283-8de3-8588ec44777c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:54.828283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-63cc039a-46c3-4283-8de3-8588ec44777c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:54.896333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1056/2127] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:41:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:54.938111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1d93-c9b6-4429-b66a-ecfac42905ef None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:54.938546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1d93-c9b6-4429-b66a-ecfac42905ef None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:54.938922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1d93-c9b6-4429-b66a-ecfac42905ef None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:54.940877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1057/2128] 10.222.0.20 () {54 vars in 961 bytes} [Tue Aug 9 02:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 02:41:54.947520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:54.947947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:54.948229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:54.997371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:54.998637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:55.005554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc039a-46c3-4283-8de3-8588ec44777c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:55.013388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:55.014385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:55.014879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:55.015569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] PATH_INFO: `/v3/users/113d48eb42524e62bfc95606f93f3736` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:55.018708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113d48eb42524e62bfc95606f93f3736)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:55.030416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:55.030616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1058/2129] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Aug 9 02:41:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:55.030616 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:55.041111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49bf93-2b20-4670-9e41-f7fb22108ab0 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:55.046082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:55.047426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:55.047862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:55.048283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:55.060164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:55.061688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:55.080349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1059/2130] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:55.081188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:55.099552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc3f2f-3e2e-44a0-9163-04a58f2c92d2 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:55.107130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:55.109101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:55.109439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:55.109674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:55.132260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:55.139970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:55.140819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113d48eb42524e62bfc95606f93f3736'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.158104 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113d48eb42524e62bfc95606f93f3736'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.158837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113d48eb42524e62bfc95606f93f3736'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.168761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1060/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:41:55.172148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113d48eb42524e62bfc95606f93f3736'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.187334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:55.205292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-289d2418-9598-49c5-a2b2-810c3a60c2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113d48eb42524e62bfc95606f93f3736'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.218585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1072/2132] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:41:54 2022] DELETE /identity/v3/users/113d48eb42524e62bfc95606f93f3736 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:55.661982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:55.675673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14fb14-5bdd-457f-8689-e8ca96c657ca service placement] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:55.759104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:55.760163 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:55.760488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:55.761577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] PATH_INFO: `/v3/projects/88f96c9ae9b44b91a4db2187160a5688` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:55.763055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f96c9ae9b44b91a4db2187160a5688)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:55.765532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:55.791703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:55.828170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f96c9ae9b44b91a4db2187160a5688', 'request_id': 'req-ffdac8f2-d50a-437c-9d71-d874f90cd72c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.849179 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f96c9ae9b44b91a4db2187160a5688', 'request_id': 'req-ffdac8f2-d50a-437c-9d71-d874f90cd72c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.864460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f96c9ae9b44b91a4db2187160a5688', 'request_id': 'req-ffdac8f2-d50a-437c-9d71-d874f90cd72c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.884814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] The token cache is being invalidate because project 88f96c9ae9b44b91a4db2187160a5688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:41:55.885109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.888240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffdac8f2-d50a-437c-9d71-d874f90cd72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:41:55.890324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1061/2133] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:41:55 2022] DELETE /identity/v3/projects/88f96c9ae9b44b91a4db2187160a5688 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:55.896460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:55.897210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:55.897210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:55.925011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:55.926558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:56.042193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1073/2134] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:56.050320 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.050632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.050888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.082175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:56.101033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:56.111423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1062/2135] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:41:56.119487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.132046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5a86-4621-42c0-a766-fe6b8cbda490 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.137377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.138014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.138325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.138631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.139298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:41:56.139600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.141550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.162418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1074/2136] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:41:56.171539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.182561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e391-5489-485a-956f-9152bb21bf0c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.187358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.188679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.188967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.189337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.190168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.193396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.236268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1063/2137] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:41:56 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:56.247451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.267100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6d88e-0027-4533-9c71-95ebd969841b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.334690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.335339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.335658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.335937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.336614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.338793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.365342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1075/2138] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:41:56 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:41:56.369748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.386711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06acc1fd-b6f2-43a1-a8cc-e4e689d37d09 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.394427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.395438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.395913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.396415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.397087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.398825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.415999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1064/2139] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:56.417169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.429313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906f2d6-e933-475a-a36d-916146293cc1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.437233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb54e7e-ea9b-439c-9cac-9eddba885da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.437945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb54e7e-ea9b-439c-9cac-9eddba885da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.438288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb54e7e-ea9b-439c-9cac-9eddba885da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.438568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb54e7e-ea9b-439c-9cac-9eddba885da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.439234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb54e7e-ea9b-439c-9cac-9eddba885da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.441091 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb54e7e-ea9b-439c-9cac-9eddba885da8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.453356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1076/2140] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:41:56.457954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.468702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b256cc-754f-450d-9929-eb550b101764 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.473907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.474615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.474899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.475174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] PATH_INFO: `/v3/projects/ca6695b7eb3b40ecb4d46292a95e6122/users/81447e83a97b4519ac6d0eac9a06bdef/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.492264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6695b7eb3b40ecb4d46292a95e6122, user_id=81447e83a97b4519ac6d0eac9a06bdef, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.495130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.514920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1065/2141] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:41:56 2022] PUT /identity/v3/projects/ca6695b7eb3b40ecb4d46292a95e6122/users/81447e83a97b4519ac6d0eac9a06bdef/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:41:56.519810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eeead-9f34-47fe-a50e-429cc395166e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.520094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eeead-9f34-47fe-a50e-429cc395166e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.522581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eeead-9f34-47fe-a50e-429cc395166e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.572008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f64eeead-9f34-47fe-a50e-429cc395166e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:41:56.572008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f64eeead-9f34-47fe-a50e-429cc395166e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:56.661787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:41:56.709219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1077/2142] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:41:56.723858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:41:56.726263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:41:56.828856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b7e2-c86e-4ec3-bad2-85b863cd7ce0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:41:56.836515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:41:56.837463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:41:56.837992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:41:56.838575 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:41:56.858029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:41:56.859958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:41:56.886410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1066/2143] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:00.167077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f52db-519a-4914-8ee2-f019b9c1d0b1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:00.167593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f52db-519a-4914-8ee2-f019b9c1d0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:00.168184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f52db-519a-4914-8ee2-f019b9c1d0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:00.235862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a85f52db-519a-4914-8ee2-f019b9c1d0b1 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:00.240340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a85f52db-519a-4914-8ee2-f019b9c1d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:00.389624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1078/2144] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:42:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:00.403951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:00.406097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:00.541172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:00.582868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28fb87-ab97-4bed-8124-b9edcf99ad83 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:00.588626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:00.589322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:00.589776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:00.590245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:00.609195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:00.610874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:00.625484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1067/2145] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:42:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:42:01.648409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bed65-91b1-4abc-a36d-9697d59a8496 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:01.648987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bed65-91b1-4abc-a36d-9697d59a8496 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:01.649380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bed65-91b1-4abc-a36d-9697d59a8496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:01.671214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-fb2bed65-91b1-4abc-a36d-9697d59a8496 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:01.672972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fb2bed65-91b1-4abc-a36d-9697d59a8496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:01.685299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1079/2146] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 9 02:42:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:01.717388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:01.728878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62ba4b-9d4d-44e8-b5c4-8f90bddad65b service swift] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:01.735441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:01.736340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:01.736837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:01.737334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:01.747551 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:01.750202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:01.762717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1068/2147] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:42:02.243116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a995-1429-4663-8cda-5ba2f6a5a8f3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:02.243920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a995-1429-4663-8cda-5ba2f6a5a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:02.244438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81a995-1429-4663-8cda-5ba2f6a5a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:02.271770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2f81a995-1429-4663-8cda-5ba2f6a5a8f3 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:02.273415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2f81a995-1429-4663-8cda-5ba2f6a5a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:02.284263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1080/2148] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:42:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:02.299211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:02.324196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b1451-b365-4438-a720-8204b2291c30 service swift] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:02.330435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e32fe-ff12-49ae-b3e4-0642870dd21b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:02.331457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e32fe-ff12-49ae-b3e4-0642870dd21b service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:02.331931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e32fe-ff12-49ae-b3e4-0642870dd21b service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:02.332400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e32fe-ff12-49ae-b3e4-0642870dd21b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:02.344539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e32fe-ff12-49ae-b3e4-0642870dd21b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:02.348042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e32fe-ff12-49ae-b3e4-0642870dd21b service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:02.365512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:02.366475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:02.366937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:02.371754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1069/2149] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:42:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:42:02.411482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] MFA Rules not processed for user `f6d6b64eb434414ab61f19b170385c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:02.413053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:02.513480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1081/2150] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:02.928836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c97a1d-d8a0-4755-b73c-bb8396a75741 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:02.929334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c97a1d-d8a0-4755-b73c-bb8396a75741 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:02.929441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c97a1d-d8a0-4755-b73c-bb8396a75741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:02.951140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-33c97a1d-d8a0-4755-b73c-bb8396a75741 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:02.953205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-33c97a1d-d8a0-4755-b73c-bb8396a75741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:02.964417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1070/2151] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 9 02:42:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:02.980537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:02.993889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbb3f0-4bef-40f2-ad51-2f1c4aaf017a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:03.004215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:03.005388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:03.006069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:03.006542 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:03.017719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:03.019801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:03.044207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1082/2152] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:42:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:42:03.548676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995dd55-79d3-4217-9b8d-4be4b9c27a8a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:03.549037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995dd55-79d3-4217-9b8d-4be4b9c27a8a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:03.549303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995dd55-79d3-4217-9b8d-4be4b9c27a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:03.600086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8995dd55-79d3-4217-9b8d-4be4b9c27a8a None None] MFA Rules not processed for user `81447e83a97b4519ac6d0eac9a06bdef`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:03.601311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8995dd55-79d3-4217-9b8d-4be4b9c27a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:03.767496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1071/2153] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:03.782484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:03.794383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:03.932550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202e6e6-d8b5-4e94-a7a1-cf898a8770e2 service swift] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:03.941834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:03.943332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:03.943862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:03.944768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:03.959081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:03.962860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:03.969738 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:03.988372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1083/2154] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:05.507679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:05.508641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:05.509194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:05.538586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:05.540347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:05.550420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1072/2155] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:42:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:05.580920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:05.600320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841e1f-8efd-41e2-8dd0-a936f7e6c452 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:05.607518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285252a-4d2c-432e-890d-4518c473b906 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:05.610699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285252a-4d2c-432e-890d-4518c473b906 service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:05.611863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285252a-4d2c-432e-890d-4518c473b906 service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:05.612375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285252a-4d2c-432e-890d-4518c473b906 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:05.634799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285252a-4d2c-432e-890d-4518c473b906 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:05.637344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285252a-4d2c-432e-890d-4518c473b906 service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:05.656212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1084/2156] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:42:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:42:06.014848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:06.016579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:06.071716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff395e-7636-4cd6-ac03-b9a94a5e5c6c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:06.088139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:06.088778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:06.089230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:06.089731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] PATH_INFO: `/v3/users/33f584ae6eae48e994c19ae3786b95d4` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:06.094277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f584ae6eae48e994c19ae3786b95d4)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:06.096510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:06.257728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f584ae6eae48e994c19ae3786b95d4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:06.259289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f584ae6eae48e994c19ae3786b95d4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:06.279396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f584ae6eae48e994c19ae3786b95d4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:06.289347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f584ae6eae48e994c19ae3786b95d4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:06.311058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f584ae6eae48e994c19ae3786b95d4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:06.315123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:06.329674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1073/2157] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:42:06 2022] DELETE /identity/v3/users/33f584ae6eae48e994c19ae3786b95d4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:06.334184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:06.334694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:06.334982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:06.376596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:06.381020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:06.513553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1085/2158] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:06.514972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:06.516480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:06.746376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c21ca6-5b01-4b56-bd4c-296d5a9dbb86 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:06.760513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e02499-c3ba-473c-aa77-44570836bf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:06.762688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02499-c3ba-473c-aa77-44570836bf05 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:06.764050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02499-c3ba-473c-aa77-44570836bf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:06.764830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02499-c3ba-473c-aa77-44570836bf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:06.791123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e02499-c3ba-473c-aa77-44570836bf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:06.795922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e02499-c3ba-473c-aa77-44570836bf05 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:06.841338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1074/2159] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:42:07.187535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:07.190514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:07.219500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e8868-3f76-4cb1-a4a0-82a41840a9f0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:07.235829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:07.236735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:07.240277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:07.240725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] PATH_INFO: `/v3/projects/017a524b579e48d2bc6fe99ab8332456` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:07.245106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017a524b579e48d2bc6fe99ab8332456)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:07.250941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:07.422255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017a524b579e48d2bc6fe99ab8332456', 'request_id': 'req-dbe9d5dc-8996-4af4-a53b-4c69658d4590'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:07.441312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017a524b579e48d2bc6fe99ab8332456', 'request_id': 'req-dbe9d5dc-8996-4af4-a53b-4c69658d4590'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:07.459333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017a524b579e48d2bc6fe99ab8332456', 'request_id': 'req-dbe9d5dc-8996-4af4-a53b-4c69658d4590'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:07.495601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] The token cache is being invalidate because project 017a524b579e48d2bc6fe99ab8332456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:42:07.496579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:07.499428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-dbe9d5dc-8996-4af4-a53b-4c69658d4590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:07.503458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1086/2160] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:42:07 2022] DELETE /identity/v3/projects/017a524b579e48d2bc6fe99ab8332456 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:07.509600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182f462-1006-4b4c-931d-7342f16959dc None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:07.510108 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182f462-1006-4b4c-931d-7342f16959dc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:07.510598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182f462-1006-4b4c-931d-7342f16959dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:07.551039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4182f462-1006-4b4c-931d-7342f16959dc None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:07.553155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4182f462-1006-4b4c-931d-7342f16959dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:07.685085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:07.685681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:07.687134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:07.766320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] MFA Rules not processed for user `81447e83a97b4519ac6d0eac9a06bdef`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:07.768302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1075/2161] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:07.777061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:07.777594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:07.778021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:07.780122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:07.820892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:07.824325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:07.849148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1076/2162] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:42:07.851904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:07.872702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cc348-2542-4676-a8e2-a996e21c6836 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:07.883570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:07.886737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:07.887254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:07.887690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:07.889001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:42:07.890270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:07.894283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:07.919757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1077/2163] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:42:07.929933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:07.962402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a8710-5884-4c37-bb06-957be1cad742 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:07.983920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:07.989764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:07.990999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:07.991671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:07.993177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:08.004626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:08.055703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1087/2164] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:42:08.152733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1078/2165] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:42:07 2022] POST /identity/v3/projects => generated 399 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:08.153867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:08.174320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e7e88-0e18-45b5-9270-50bacc8e64c3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:08.233430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:08.342789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:08.343084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:08.343703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:08.344047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:08.344643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:08.346814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:08.377093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1088/2166] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:42:08 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:08.382771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:08.401878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e31697-48f8-4e2b-9444-3ec1ea0e6692 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:08.421907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:08.422774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:08.423709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:08.424137 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:08.424985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:08.429234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:08.444079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1079/2167] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:08.450167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:08.465305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5fa6b-1eb6-4388-b0b5-77393821425e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:08.472288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7965ae-e203-4ee4-a32f-1db514f49357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:08.473143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7965ae-e203-4ee4-a32f-1db514f49357 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:08.473530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7965ae-e203-4ee4-a32f-1db514f49357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:08.474465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7965ae-e203-4ee4-a32f-1db514f49357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:08.475583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7965ae-e203-4ee4-a32f-1db514f49357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:08.478206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7965ae-e203-4ee4-a32f-1db514f49357 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:08.495012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1089/2168] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:08.501894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:08.518251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7df238-53f8-4943-ba08-44bf4d7aa0aa admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:08.526836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:08.528669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:08.529682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:08.530228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] PATH_INFO: `/v3/projects/8937041c1e8a4c9898abed6e3211b20c/users/08286274c85c4c56acaef3998c348240/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:08.557815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8937041c1e8a4c9898abed6e3211b20c, user_id=08286274c85c4c56acaef3998c348240, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:08.564135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:08.591622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1080/2169] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:42:08 2022] PUT /identity/v3/projects/8937041c1e8a4c9898abed6e3211b20c/users/08286274c85c4c56acaef3998c348240/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:08.610550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:08.611008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:08.611469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:08.666144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] MFA Rules not processed for user `08286274c85c4c56acaef3998c348240`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:08.668663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:08.866132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1090/2170] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:08.883512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:08.885783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:08.993580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:09.059330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb592e49-012f-4a93-bd0b-6faf0c0cda38 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:09.066682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c0b7e-445a-42a4-a96b-86463ea840e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:09.068209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c0b7e-445a-42a4-a96b-86463ea840e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:09.068981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c0b7e-445a-42a4-a96b-86463ea840e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:09.069952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c0b7e-445a-42a4-a96b-86463ea840e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:09.084622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c0b7e-445a-42a4-a96b-86463ea840e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:09.087759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c0b7e-445a-42a4-a96b-86463ea840e2 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:09.102640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1081/2171] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:09.164165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:09.166728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:09.316827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d85e8-81c9-435e-80ba-f6fa9ff60ad5 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:09.334914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:09.336069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:09.336595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:09.337358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] PATH_INFO: `/v3/users/a0f110e337974435bd065c5995234fe3` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:09.339867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f110e337974435bd065c5995234fe3)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:09.344210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:09.405131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:09.459937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f110e337974435bd065c5995234fe3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:09.480616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f110e337974435bd065c5995234fe3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:09.481408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f110e337974435bd065c5995234fe3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:09.492877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f110e337974435bd065c5995234fe3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:09.507572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f110e337974435bd065c5995234fe3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:09.518201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1091/2172] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:42:09 2022] DELETE /identity/v3/users/a0f110e337974435bd065c5995234fe3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:09.521202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:09.521463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:09.521738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:09.562864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:09.567797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:09.754613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:09.756395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:09.759070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1082/2173] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:09.923428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff9f4c-555c-4b90-b7f2-49d1302a6101 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:09.927737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b6447-6970-4c43-85a0-b45d035fbdc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:09.929381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b6447-6970-4c43-85a0-b45d035fbdc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:09.930186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b6447-6970-4c43-85a0-b45d035fbdc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:09.930186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b6447-6970-4c43-85a0-b45d035fbdc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:09.949744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b6447-6970-4c43-85a0-b45d035fbdc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:09.967344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b6447-6970-4c43-85a0-b45d035fbdc0 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:10.014819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1092/2174] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:42:10.876974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:10.879003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:10.908015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebd4c1-38bc-4e88-b3fa-fda685306486 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:10.929044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:10.930124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:10.930643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:10.931213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] PATH_INFO: `/v3/projects/9cd21f96d5df4ba2afe15aa143648788` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:10.933316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd21f96d5df4ba2afe15aa143648788)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:10.936378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:11.093331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd21f96d5df4ba2afe15aa143648788', 'request_id': 'req-5bd17389-74cb-450c-9a2c-5923dbada18e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:11.109831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd21f96d5df4ba2afe15aa143648788', 'request_id': 'req-5bd17389-74cb-450c-9a2c-5923dbada18e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:11.136535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd21f96d5df4ba2afe15aa143648788', 'request_id': 'req-5bd17389-74cb-450c-9a2c-5923dbada18e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:11.175466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] The token cache is being invalidate because project 9cd21f96d5df4ba2afe15aa143648788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:42:11.175466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:11.176862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5bd17389-74cb-450c-9a2c-5923dbada18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:11.185573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1083/2175] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:42:10 2022] DELETE /identity/v3/projects/9cd21f96d5df4ba2afe15aa143648788 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:11.192287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:11.193026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:11.193317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:11.273327 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:11.286948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:11.423514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1093/2176] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:11.433611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:11.434190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:11.434617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:11.477016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:11.479130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:11.502203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1084/2177] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:11.508156 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:11.525080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d2215-4678-4fc1-8e5b-312ea170715a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:11.537028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:11.538873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:11.540666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:11.541095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:11.542708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:42:11.543786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:11.546221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:11.598963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1094/2178] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:42:11.601251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:11.612648 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352f39-64c8-480f-bf83-6fe91c2991c5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:11.628434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:11.629047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:11.629332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:11.629633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:11.630314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:11.632363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:11.686833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1085/2179] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:42:11 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:11.690265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:11.718552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8344cc1-3f0c-4c2e-81dc-1a9c0a662272 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:11.808615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:11.905026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:11.908934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:11.909967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:11.911706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:11.913694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:11.920538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:12.005831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1095/2180] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:42:11 2022] POST /identity/v3/users => generated 282 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:12.007022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:12.025841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93588-cc85-4439-8655-3328ef331090 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:12.042842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:12.043657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:12.043970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:12.044264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:12.044974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:12.046811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:12.080745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1086/2181] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:42:12.082630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:12.097117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c87d3e-f98b-443b-b0a2-b4e1f38f402c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:12.120945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b2803-d532-4b8a-8456-e1d369a51b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:12.122416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b2803-d532-4b8a-8456-e1d369a51b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:12.123014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b2803-d532-4b8a-8456-e1d369a51b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:12.123555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b2803-d532-4b8a-8456-e1d369a51b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:12.128147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b2803-d532-4b8a-8456-e1d369a51b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:12.131572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b2803-d532-4b8a-8456-e1d369a51b07 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:12.161445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1096/2182] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:12.162733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:12.176444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdc8b7-b48a-4610-9235-1f9375e38829 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:12.188419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:12.189779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:12.190198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:12.190590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] PATH_INFO: `/v3/projects/3367a4131e2c4948ba9e9d25943cb865/users/72d3a25d70a4455a8f581c6abc595777/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:12.207507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:12.216705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3367a4131e2c4948ba9e9d25943cb865, user_id=72d3a25d70a4455a8f581c6abc595777, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:12.220471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:12.251394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1087/2183] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:42:12 2022] PUT /identity/v3/projects/3367a4131e2c4948ba9e9d25943cb865/users/72d3a25d70a4455a8f581c6abc595777/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:12.253817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71fee5-648d-42bd-bc5f-f265c06c954b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:12.254793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71fee5-648d-42bd-bc5f-f265c06c954b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:12.255072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71fee5-648d-42bd-bc5f-f265c06c954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:12.287771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2c71fee5-648d-42bd-bc5f-f265c06c954b None None] MFA Rules not processed for user `72d3a25d70a4455a8f581c6abc595777`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:12.289116 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2c71fee5-648d-42bd-bc5f-f265c06c954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:12.393911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1097/2184] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:42:12.400555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:12.402353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:12.499331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd8221-ee10-4f2f-a165-51d21d9b184f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:12.504146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb7e14-53d2-4d88-91ca-a974e40fb491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:12.504802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb7e14-53d2-4d88-91ca-a974e40fb491 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:12.505086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb7e14-53d2-4d88-91ca-a974e40fb491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:12.505502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb7e14-53d2-4d88-91ca-a974e40fb491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:12.522109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb7e14-53d2-4d88-91ca-a974e40fb491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:12.523792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb7e14-53d2-4d88-91ca-a974e40fb491 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:12.534892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1088/2185] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:13.421481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a875-4458-4733-a9e8-2b200c907de1 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:13.421961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a875-4458-4733-a9e8-2b200c907de1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:13.422143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27a875-4458-4733-a9e8-2b200c907de1 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:13.423562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1098/2186] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:42:13.450932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:13.451532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:13.451532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:13.474612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] MFA Rules not processed for user `dea38c5b41e5432d9f94f3248824e262`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:13.475955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:13.556362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1089/2187] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:42:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:13.570076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27a875-4458-4733-a9e8-2b200c907de1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:13.574045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8a27a875-4458-4733-a9e8-2b200c907de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:13.595541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27a875-4458-4733-a9e8-2b200c907de1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:13.601663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:13.606289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:13.606289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:13.606289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:13.618834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:13.620622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:13.632177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1099/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:14.565803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:14.575916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:14.667768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db9bb5-4ea9-49f6-9f61-c589bc3b5c6e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:14.672530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:14.673189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:14.673469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:14.673743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] PATH_INFO: `/v3/users/e0165ad1bd7147e1a9dded684004208f` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:14.674950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0165ad1bd7147e1a9dded684004208f)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:14.676907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:14.718908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:14.735955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0165ad1bd7147e1a9dded684004208f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:14.736638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0165ad1bd7147e1a9dded684004208f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:14.745309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0165ad1bd7147e1a9dded684004208f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:14.753249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0165ad1bd7147e1a9dded684004208f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:14.767161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0165ad1bd7147e1a9dded684004208f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:14.773463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1090/2189] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:42:14 2022] DELETE /identity/v3/users/e0165ad1bd7147e1a9dded684004208f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:15.022875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:15.046874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568faa-deca-4d3c-9289-57f5c633e930 service placement] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:15.111377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:15.112024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:15.112308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:15.112584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] PATH_INFO: `/v3/projects/eb576abacf4a41b59056aeb0b683225c` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:15.113710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb576abacf4a41b59056aeb0b683225c)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:15.115683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:15.158763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:15.175005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb576abacf4a41b59056aeb0b683225c', 'request_id': 'req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.183318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb576abacf4a41b59056aeb0b683225c', 'request_id': 'req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.189745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb576abacf4a41b59056aeb0b683225c', 'request_id': 'req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.200001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] The token cache is being invalidate because project eb576abacf4a41b59056aeb0b683225c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:42:15.200292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.200944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.202507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1100/2190] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:42:15 2022] DELETE /identity/v3/projects/eb576abacf4a41b59056aeb0b683225c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:15.206699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:15.208474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:15.223047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286658cd-e5b8-437d-8ba7-2ae8343b5239 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:15.308902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:15.309887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:15.310295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:15.310583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] PATH_INFO: `/v3/users/c48a57cfd87f43b992d1ad55c24898e4` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:15.311870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48a57cfd87f43b992d1ad55c24898e4)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:15.314082 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:15.372744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48a57cfd87f43b992d1ad55c24898e4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.373498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48a57cfd87f43b992d1ad55c24898e4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.402746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48a57cfd87f43b992d1ad55c24898e4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.415002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48a57cfd87f43b992d1ad55c24898e4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.432921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-195c6641-5a79-44c7-83bf-b12c080a61e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48a57cfd87f43b992d1ad55c24898e4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.447907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1091/2191] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:42:15 2022] DELETE /identity/v3/users/c48a57cfd87f43b992d1ad55c24898e4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:15.730636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:15.732191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:15.752415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e89fd9-7c96-47fb-80e2-147e6b8f2004 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:15.866370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:15.867172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:15.867553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:15.868086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] PATH_INFO: `/v3/projects/97312962054e4963bc241a74a8142752` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:15.869448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97312962054e4963bc241a74a8142752)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:15.871856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:15.944767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97312962054e4963bc241a74a8142752', 'request_id': 'req-55754f68-ac85-4f48-bb8c-13a953cf8560'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.954081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97312962054e4963bc241a74a8142752', 'request_id': 'req-55754f68-ac85-4f48-bb8c-13a953cf8560'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.964186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97312962054e4963bc241a74a8142752', 'request_id': 'req-55754f68-ac85-4f48-bb8c-13a953cf8560'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.985863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] The token cache is being invalidate because project 97312962054e4963bc241a74a8142752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:42:15.986579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:15.998791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-55754f68-ac85-4f48-bb8c-13a953cf8560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:16.000891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1101/2192] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:42:15 2022] DELETE /identity/v3/projects/97312962054e4963bc241a74a8142752 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:16.018622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.019135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.019308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.061970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:16.065489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:16.132287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:16.208625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.209147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.209147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.217846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1092/2193] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:16.232887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:16.234328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:16.244063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1102/2194] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:42:16.248892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.259433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279920-77c9-428e-a155-c2a4a8efc1d6 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.277741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.279245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.279797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.280277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.283454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:42:16.283988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.286370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.302474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1093/2195] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:42:16.306854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.318305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2413e-4475-4433-9d6d-9644c1ee8f2b None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.324352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.325090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.325446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.325800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.326550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.328602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.382187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1103/2196] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:42:16 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:42:16.386364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.398708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b82ff-ce68-4e40-80be-0b65931daa66 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.493756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.494478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.494886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.495009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.495856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.497859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.522594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1094/2197] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:42:16 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:16.526762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.537503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65438e5-d3d3-4da0-a837-9e6b62e2b4d3 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.542322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.542936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.543216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.543493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.544168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.545786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.562243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1104/2198] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:42:16.564053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.577277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07208d7-5b19-48fe-add5-33708f91b5e8 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.584601 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bbb2d-1147-4b76-82be-3169485835c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.585776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bbb2d-1147-4b76-82be-3169485835c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.586950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bbb2d-1147-4b76-82be-3169485835c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.587247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bbb2d-1147-4b76-82be-3169485835c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.587981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bbb2d-1147-4b76-82be-3169485835c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.589928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bbb2d-1147-4b76-82be-3169485835c5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.606277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1095/2199] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:16.611145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.625163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ac4b-85b9-43e5-bf34-f3cacf9631c1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.632558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.633451 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.633698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.634604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] PATH_INFO: `/v3/projects/92b7075fdda741489109437940f40cda/users/a3a9579a0fc2476e9b516f8cc26af832/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.639060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:16.654592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b7075fdda741489109437940f40cda, user_id=a3a9579a0fc2476e9b516f8cc26af832, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.657471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.686305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1105/2200] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:42:16 2022] PUT /identity/v3/projects/92b7075fdda741489109437940f40cda/users/a3a9579a0fc2476e9b516f8cc26af832/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:42:16.689921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.690246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.690549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.726901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] MFA Rules not processed for user `a3a9579a0fc2476e9b516f8cc26af832`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:16.728328 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:16.817887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1096/2201] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:16.831260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.833766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:16.895979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1143-3bbe-42f8-be3c-0b329cd81fd6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:16.899922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e33c3-5a97-4268-b7e0-72144eb10f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:16.900632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e33c3-5a97-4268-b7e0-72144eb10f26 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:16.900978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e33c3-5a97-4268-b7e0-72144eb10f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:16.901380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e33c3-5a97-4268-b7e0-72144eb10f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:16.909972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e33c3-5a97-4268-b7e0-72144eb10f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:16.911515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e33c3-5a97-4268-b7e0-72144eb10f26 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:16.920690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1106/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:16.947402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:16.973983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:17.013778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3b1bc-4291-4967-abf2-ff55203f5edb None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:17.019000 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:17.019788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:17.019893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:17.020148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] PATH_INFO: `/v3/users/a3a9579a0fc2476e9b516f8cc26af832` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:17.021525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a9579a0fc2476e9b516f8cc26af832)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:17.023541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:17.076687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a9579a0fc2476e9b516f8cc26af832'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.082445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a9579a0fc2476e9b516f8cc26af832'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.092684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a9579a0fc2476e9b516f8cc26af832'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.104105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a9579a0fc2476e9b516f8cc26af832'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.120161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a9579a0fc2476e9b516f8cc26af832'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.128949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1097/2203] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:42:16 2022] DELETE /identity/v3/users/a3a9579a0fc2476e9b516f8cc26af832 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:42:17.134224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:17.134520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:17.134796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:17.157843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:17.159469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:17.277910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1107/2204] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:17.289075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:17.290821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:17.396905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bdb15-95e3-43ef-8928-1487b36bedcc admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:17.400624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:17.401251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:17.401521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:17.401811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:17.419025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:17.420492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:17.430989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1098/2205] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:17.881729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:17.883333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:17.897004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6324cdf-094b-45e5-8d20-f205c6733888 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:17.901886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:17.902585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:17.902862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:17.903139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] PATH_INFO: `/v3/projects/92b7075fdda741489109437940f40cda` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:17.904290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b7075fdda741489109437940f40cda)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:17.906251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:17.969464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b7075fdda741489109437940f40cda', 'request_id': 'req-22ef72f8-8717-4bf9-a7e8-39530aad271b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.981437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b7075fdda741489109437940f40cda', 'request_id': 'req-22ef72f8-8717-4bf9-a7e8-39530aad271b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:17.990182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b7075fdda741489109437940f40cda', 'request_id': 'req-22ef72f8-8717-4bf9-a7e8-39530aad271b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.001891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] The token cache is being invalidate because project 92b7075fdda741489109437940f40cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:42:18.002196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.003003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-22ef72f8-8717-4bf9-a7e8-39530aad271b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.006274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1108/2206] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:42:17 2022] DELETE /identity/v3/projects/92b7075fdda741489109437940f40cda => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:18.231425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:18.233414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:18.249889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783f4c8-6c79-433b-90c2-cbe354c33918 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:18.295478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:18.312581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:18.313226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:18.313506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:18.313803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] PATH_INFO: `/v3/users/08286274c85c4c56acaef3998c348240` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:18.314966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08286274c85c4c56acaef3998c348240)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:18.316913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:18.370066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08286274c85c4c56acaef3998c348240'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.370790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08286274c85c4c56acaef3998c348240'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.379063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08286274c85c4c56acaef3998c348240'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.388469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08286274c85c4c56acaef3998c348240'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.402225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08286274c85c4c56acaef3998c348240'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.413733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:18.414318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:18.414318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:18.417075 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1099/2207] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:42:18 2022] DELETE /identity/v3/users/08286274c85c4c56acaef3998c348240 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:18.434543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:18.435813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:18.441277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:18.499332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1109/2208] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:42:18.509509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:18.510960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:18.576527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e38eb9-0da2-454a-bc48-c6f9b99dae09 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:18.581438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a8b61-f71d-487a-8b9b-7b744872e063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:18.582325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8b61-f71d-487a-8b9b-7b744872e063 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:18.582670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8b61-f71d-487a-8b9b-7b744872e063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:18.583046 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8b61-f71d-487a-8b9b-7b744872e063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:18.597779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a8b61-f71d-487a-8b9b-7b744872e063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:18.599428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a8b61-f71d-487a-8b9b-7b744872e063 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:18.610222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1100/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:42:18.897419 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:18.899111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:18.911955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ec4e9-79bd-410f-8834-5c1a5b2474ca None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:18.923302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:18.924227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:18.924510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:18.924780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] PATH_INFO: `/v3/projects/8937041c1e8a4c9898abed6e3211b20c` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:18.926454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8937041c1e8a4c9898abed6e3211b20c)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:18.928033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:18.984883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8937041c1e8a4c9898abed6e3211b20c', 'request_id': 'req-2a45ada6-2d04-4a01-a84b-7980034a8672'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:18.993627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8937041c1e8a4c9898abed6e3211b20c', 'request_id': 'req-2a45ada6-2d04-4a01-a84b-7980034a8672'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:19.000794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8937041c1e8a4c9898abed6e3211b20c', 'request_id': 'req-2a45ada6-2d04-4a01-a84b-7980034a8672'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:19.012828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] The token cache is being invalidate because project 8937041c1e8a4c9898abed6e3211b20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:42:19.013244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:19.013926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2a45ada6-2d04-4a01-a84b-7980034a8672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:42:19.015735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1110/2210] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:42:18 2022] DELETE /identity/v3/projects/8937041c1e8a4c9898abed6e3211b20c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:19.022340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.022633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.022913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.043559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:19.044944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:19.120889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1101/2211] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:19.124398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.124866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.125215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.152512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:19.154140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:19.166202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1111/2212] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:19.170922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.181361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1e641-58f8-4b33-97b8-fba622e2b7e3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.186408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.187035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.187319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.187729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.188396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:42:19.188705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.190501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.192509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:19.212794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1102/2213] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:42:19.214242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.224129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43315d75-bee4-434a-a715-1f2d2dcca3f4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.230632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.231495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.231829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.232106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.233023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.235212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.280345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1112/2214] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:42:19 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:19.281963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.292669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49d7b4-b9fe-4dd4-9eb0-cddfd21ea005 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.354883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.355854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.356129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.356395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.357024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.358983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.382402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1103/2215] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:42:19 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:19.386809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.397372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226714c4-3808-4d2d-83fe-7a1f775b9676 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.402418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.403052 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.403336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.403618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.404268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.405887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.417097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1113/2216] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:19.421419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.432316 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddcb38-d67b-49d6-8e1b-6d2686cc9769 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.437421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f9fb4-c4b0-480e-bd20-bdade728cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.438019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f9fb4-c4b0-480e-bd20-bdade728cab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.438545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f9fb4-c4b0-480e-bd20-bdade728cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.438671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f9fb4-c4b0-480e-bd20-bdade728cab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.439274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f9fb4-c4b0-480e-bd20-bdade728cab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.440846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f9fb4-c4b0-480e-bd20-bdade728cab0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.451161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1104/2217] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:19.456267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.466613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2e7b-d516-4426-8cbd-6f3ccfa96e34 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.471579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.472245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.472519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.472790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] PATH_INFO: `/v3/projects/9aadaf242f674436a45eae700fb52fc1/users/90fa94f90d49485a81d502929bcd0b93/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.488964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aadaf242f674436a45eae700fb52fc1, user_id=90fa94f90d49485a81d502929bcd0b93, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.491740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.505926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1114/2218] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:42:19 2022] PUT /identity/v3/projects/9aadaf242f674436a45eae700fb52fc1/users/90fa94f90d49485a81d502929bcd0b93/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:19.511346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf484061-08fe-4f45-aaa2-812f806ca674 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.511632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf484061-08fe-4f45-aaa2-812f806ca674 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.511910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf484061-08fe-4f45-aaa2-812f806ca674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.531473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-bf484061-08fe-4f45-aaa2-812f806ca674 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:19.532794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-bf484061-08fe-4f45-aaa2-812f806ca674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:19.594593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1105/2219] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:19.604525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:19.607124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:19.640827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:19.665095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50031051-f071-411f-8529-1b22fd063464 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:19.669302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:19.669911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:19.670222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:19.670497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:19.680309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:19.681866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:19.691694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1115/2220] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:25.459097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:25.459097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:25.459097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:25.496132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] MFA Rules not processed for user `90fa94f90d49485a81d502929bcd0b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:25.497371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:25.598281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1106/2221] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:25.602787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:25.615013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd8f7d-18d4-48f6-90a5-172cfbc6bbcc service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:25.622015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:25.622756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:25.623113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:25.623458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:25.624358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:25.628005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:25.663763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1116/2222] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:42:25 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:42:25.668277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:25.674219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:25.682256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197577a-2df6-4c25-8453-8ceba305d120 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:25.760134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:25.761021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:25.761377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:25.761724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:25.762860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:25.765022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:25.792251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1107/2223] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:42:25 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:42:25.795896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:25.810475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bc035-522d-47c4-aa92-745b97d70bc4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:25.816285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:25.816993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:25.817341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:25.817684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:25.818461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:25.820389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:25.835932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1117/2224] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:42:25.837291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:25.871205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65641b09-e910-4d69-8407-8bf6bf9d26c7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:25.879051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:25.879860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:25.880265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:25.880835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] PATH_INFO: `/v3/projects/8f7fa836383e4ffaa6209e73ed6d946a/users/e6a58dddb41a477fb1552e9a0deb01b6/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:25.904944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7fa836383e4ffaa6209e73ed6d946a, user_id=e6a58dddb41a477fb1552e9a0deb01b6, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:25.909321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:25.942330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1108/2225] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:42:25 2022] PUT /identity/v3/projects/8f7fa836383e4ffaa6209e73ed6d946a/users/e6a58dddb41a477fb1552e9a0deb01b6/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:42:25.947110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:25.986955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaf2c7-01ab-4873-afd0-862a311c3e4f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:26.181997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea75a3-9b53-49d9-a042-59f39b10198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:26.182812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea75a3-9b53-49d9-a042-59f39b10198c admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:26.183203 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea75a3-9b53-49d9-a042-59f39b10198c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:26.183580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea75a3-9b53-49d9-a042-59f39b10198c admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:26.184257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea75a3-9b53-49d9-a042-59f39b10198c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:26.186062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea75a3-9b53-49d9-a042-59f39b10198c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:26.200216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1118/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:42:26.203190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:26.218724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea2ad0-b5b9-4556-acd6-2761199447ee admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:26.223926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4976444-021c-4a0a-8184-da40bc5ef74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:26.224730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4976444-021c-4a0a-8184-da40bc5ef74c admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:26.225007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4976444-021c-4a0a-8184-da40bc5ef74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:26.225279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4976444-021c-4a0a-8184-da40bc5ef74c admin admin] PATH_INFO: `/v3/projects/8f7fa836383e4ffaa6209e73ed6d946a/users/e6a58dddb41a477fb1552e9a0deb01b6/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:26.227850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4976444-021c-4a0a-8184-da40bc5ef74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7fa836383e4ffaa6209e73ed6d946a, user_id=e6a58dddb41a477fb1552e9a0deb01b6, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:26.231598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4976444-021c-4a0a-8184-da40bc5ef74c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:26.245986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1109/2227] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:42:26 2022] PUT /identity/v3/projects/8f7fa836383e4ffaa6209e73ed6d946a/users/e6a58dddb41a477fb1552e9a0deb01b6/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:42:31.654850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:31.655347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:31.656409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:31.688435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] MFA Rules not processed for user `43232b9b191047f9a3e6ebab7ad36f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:31.690803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:31.781375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1119/2228] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:34.825079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:34.825668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:34.825668 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:34.860626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] MFA Rules not processed for user `e6a58dddb41a477fb1552e9a0deb01b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:42:34.862798 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:34.963808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1110/2229] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:42:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:42:35.067974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:35.069409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:35.137778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:35.170458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47a86-09cb-416c-9d40-a9dda04dde33 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:35.176227 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:35.176875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:35.177158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:35.177430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:35.238292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:35.238480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:35.248130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1120/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:44.960481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:45.258726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=37 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:42:47.374765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:42:47.376386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:47.439031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f8ebc-36cf-460c-938a-3ebb6688c717 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:42:47.442647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:42:47.443291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:42:47.443570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:42:47.443842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:42:47.445537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:42:47.509996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:42:47.511524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bace4e-784c-45ce-99cc-c8e5a5daf7bf service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:42:47.521709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1111/2231] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:42:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:42:57.517033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=50 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:01.880391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:01.882171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:01.958416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e30d-7013-4d15-94f7-2d52d1b8018a service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:01.963108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:01.963741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:01.964019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:01.964288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] PATH_INFO: `/v3/users/72d3a25d70a4455a8f581c6abc595777` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:01.965436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d3a25d70a4455a8f581c6abc595777)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:01.967403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:02.017015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d3a25d70a4455a8f581c6abc595777'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.025276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d3a25d70a4455a8f581c6abc595777'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.025944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d3a25d70a4455a8f581c6abc595777'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.033385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d3a25d70a4455a8f581c6abc595777'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.045743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d3a25d70a4455a8f581c6abc595777'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.051702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1121/2232] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:43:01 2022] DELETE /identity/v3/users/72d3a25d70a4455a8f581c6abc595777 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:43:02.056299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.056765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.057057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.076658 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:02.078021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:02.140753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1112/2233] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:02.150907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:02.152719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:02.214390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50745e75-d96e-4306-a856-35e9d5954f4e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:02.219546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b4f7b-69e4-4dd6-86c6-767ed4a90295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:02.220210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b4f7b-69e4-4dd6-86c6-767ed4a90295 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.220488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b4f7b-69e4-4dd6-86c6-767ed4a90295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.220755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b4f7b-69e4-4dd6-86c6-767ed4a90295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.232452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b4f7b-69e4-4dd6-86c6-767ed4a90295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:02.234631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b4f7b-69e4-4dd6-86c6-767ed4a90295 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:02.245497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1122/2234] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:43:02.567936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:02.569721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:02.583498 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9029999-567f-4ff1-81e2-5e34333e70ff None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:02.590368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:02.591027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.591305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.591570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] PATH_INFO: `/v3/projects/3367a4131e2c4948ba9e9d25943cb865` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.592768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3367a4131e2c4948ba9e9d25943cb865)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:02.595156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:02.655346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3367a4131e2c4948ba9e9d25943cb865', 'request_id': 'req-cd6289d1-774d-4da0-b1f4-2a8a761985b1'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.663960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3367a4131e2c4948ba9e9d25943cb865', 'request_id': 'req-cd6289d1-774d-4da0-b1f4-2a8a761985b1'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.672984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3367a4131e2c4948ba9e9d25943cb865', 'request_id': 'req-cd6289d1-774d-4da0-b1f4-2a8a761985b1'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.695198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] The token cache is being invalidate because project 3367a4131e2c4948ba9e9d25943cb865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:43:02.695475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.696129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd6289d1-774d-4da0-b1f4-2a8a761985b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:43:02.697729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1113/2235] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:43:02 2022] DELETE /identity/v3/projects/3367a4131e2c4948ba9e9d25943cb865 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:43:02.703067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.703413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.703832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.724556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:02.726133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:02.745834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:02.803534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1123/2236] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:02.811594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.811972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.812279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.840548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:02.841841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:02.851207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1114/2237] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:02.855331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:02.865852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72534bf0-ca73-4c24-88bb-865dd8542b44 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:02.876796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:02.878322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.878664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.878948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.879744 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:43:02.880335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:02.881964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:02.898463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1124/2238] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:43:02.902520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:02.912195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27c4b8-1926-462d-a528-760942173b7b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:02.917313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:02.917940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:02.918336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:02.918560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:02.919295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:02.921083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:02.954299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1115/2239] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:43:02 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:43:02.957523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:02.969844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ce999-731e-4ced-9010-413e006bf0d0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:03.031878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:03.032545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.032840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.033119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.033743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:03.035708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:03.065044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1125/2240] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:43:02 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:43:03.066451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:03.077371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1370-4005-4fc1-99b9-3d9d92b76bb1 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:03.082937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:03.083722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.084011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.084277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.084930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:03.086635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:03.102812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1116/2241] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:43:03.104141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:03.121193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785fcc3-3367-49b6-9425-6afaf7247c0f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:03.126635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf2599-56fd-47ef-8343-9863e153488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:03.127290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf2599-56fd-47ef-8343-9863e153488d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.127571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf2599-56fd-47ef-8343-9863e153488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.127841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf2599-56fd-47ef-8343-9863e153488d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.128518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf2599-56fd-47ef-8343-9863e153488d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:03.130265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf2599-56fd-47ef-8343-9863e153488d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:03.142990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1126/2242] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:43:03.147135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:03.158034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadbe0a-3eec-498f-bdf5-2ce43d0fc105 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:03.187578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe807-ae3c-4f35-a867-17655396bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:03.188272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe807-ae3c-4f35-a867-17655396bc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.188855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe807-ae3c-4f35-a867-17655396bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.188855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe807-ae3c-4f35-a867-17655396bc92 admin admin] PATH_INFO: `/v3/projects/2c5d24d2b8dc4844a48a76ed2ea2cffe/users/f96475caac28419eb0b7dd78375e9e52/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.205512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abe807-ae3c-4f35-a867-17655396bc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5d24d2b8dc4844a48a76ed2ea2cffe, user_id=f96475caac28419eb0b7dd78375e9e52, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:03.208583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abe807-ae3c-4f35-a867-17655396bc92 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:03.223407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1117/2243] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:43:03 2022] PUT /identity/v3/projects/2c5d24d2b8dc4844a48a76ed2ea2cffe/users/f96475caac28419eb0b7dd78375e9e52/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:43:03.228543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.228833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.229105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.235550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734c06-cd43-4471-a16b-97aa1d886b7a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.236094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734c06-cd43-4471-a16b-97aa1d886b7a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.236549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734c06-cd43-4471-a16b-97aa1d886b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.252039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:03.253367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:03.257109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c7734c06-cd43-4471-a16b-97aa1d886b7a None None] MFA Rules not processed for user `90fa94f90d49485a81d502929bcd0b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:03.258350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c7734c06-cd43-4471-a16b-97aa1d886b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:03.272660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:03.319804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1127/2244] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:03.328957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1118/2245] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:03.329408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:03.330908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:03.370251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:03.407506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92081ed-5dae-45de-bd48-ce82ed88356c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:03.413732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f2330-de3e-4927-848b-064e80d922f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:03.414390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f2330-de3e-4927-848b-064e80d922f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:03.414669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f2330-de3e-4927-848b-064e80d922f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:03.415737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f2330-de3e-4927-848b-064e80d922f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:03.425392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f2330-de3e-4927-848b-064e80d922f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:03.426880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f2330-de3e-4927-848b-064e80d922f8 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:03.437552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1128/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:43:04.690678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:04.691229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:04.691229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:04.714346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] MFA Rules not processed for user `192a8abd4c7a4535a42bea7648535dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:04.716832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:04.805342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1119/2247] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:08.546772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:08.549270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:08.551064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:08.626734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] MFA Rules not processed for user `f96475caac28419eb0b7dd78375e9e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:08.628713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:08.726621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1129/2248] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:43:08.732172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:08.747617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f08307-0d80-45f8-804a-2c608085b68b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:08.755590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:08.756363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:08.756649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:08.756928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:08.757565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:08.760535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:08.799991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1120/2249] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:43:08 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:43:08.804250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:08.816917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bd9eb-51fd-45ee-858f-a6692bc32360 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:08.887933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:08.888571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:08.888856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:08.889128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:08.890250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:08.892310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:08.919527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1130/2250] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:43:08 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:43:08.924158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:08.936363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8fbc-9925-4349-9a8e-03ae7a935e1e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:08.941522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:08.942182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:08.942504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:08.942784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:08.943448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:08.945088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:08.957642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1121/2251] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:43:08.961803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:08.974398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d2063-ccef-4ae7-ad38-f69623060bd9 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:08.979366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:08.979987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:08.980271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:08.980552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] PATH_INFO: `/v3/projects/d2e79b76b02a448f921991e7a0188871/users/cd87cc44a7a84e36b26596504f2d1cb6/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:08.997379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e79b76b02a448f921991e7a0188871, user_id=cd87cc44a7a84e36b26596504f2d1cb6, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:09.000140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:09.019250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1131/2252] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:43:08 2022] PUT /identity/v3/projects/d2e79b76b02a448f921991e7a0188871/users/cd87cc44a7a84e36b26596504f2d1cb6/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:43:09.020379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:09.031685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce93dd-b0ac-4fb6-809c-cce4ca9a5073 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:09.114350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbdcc7-3285-43ad-917b-ecb1987c214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:09.115191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbdcc7-3285-43ad-917b-ecb1987c214e admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:09.115587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbdcc7-3285-43ad-917b-ecb1987c214e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:09.116063 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbdcc7-3285-43ad-917b-ecb1987c214e admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:09.117439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbdcc7-3285-43ad-917b-ecb1987c214e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:09.118636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbdcc7-3285-43ad-917b-ecb1987c214e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:09.133047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1122/2253] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:43:09.137978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:09.155054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcaf37-8d68-411c-a141-7689849e0c73 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:09.160827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:09.161531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:09.161820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:09.162105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] PATH_INFO: `/v3/projects/d2e79b76b02a448f921991e7a0188871/users/cd87cc44a7a84e36b26596504f2d1cb6/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:09.164153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e79b76b02a448f921991e7a0188871, user_id=cd87cc44a7a84e36b26596504f2d1cb6, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:09.167070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:09.196789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1132/2254] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:43:09 2022] PUT /identity/v3/projects/d2e79b76b02a448f921991e7a0188871/users/cd87cc44a7a84e36b26596504f2d1cb6/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:43:14.878817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79161b-60bd-40b1-b435-995509cfd3dc None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:14.879760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79161b-60bd-40b1-b435-995509cfd3dc None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:14.880659 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79161b-60bd-40b1-b435-995509cfd3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:14.915326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-9a79161b-60bd-40b1-b435-995509cfd3dc None None] MFA Rules not processed for user `cd87cc44a7a84e36b26596504f2d1cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:14.917011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9a79161b-60bd-40b1-b435-995509cfd3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:14.938564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:14.999807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1123/2255] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:15.013392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:15.015348 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:15.033857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:15.099889 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc190d-c59d-4e31-9dcb-9a23e1af864a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:15.105399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:15.106013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:15.106324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:15.106967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:15.163492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:15.165025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:15.174261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1133/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:43:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:43:16.257643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:16.258143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:16.258265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:16.284511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] MFA Rules not processed for user `f96475caac28419eb0b7dd78375e9e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:43:16.286639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:16.300003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1124/2257] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:43:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:43:25.170174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=50 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:25.797450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:25.863696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75b0b-2d67-4c04-b26c-df7e3726ac77 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:25.870759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:25.871569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:25.871991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:25.872280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:25.875516 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:25.947630 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:25.949087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:25.958833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1134/2258] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:43:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:43:26.278820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=22 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:35.954407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=50 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:39.553538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:39.555669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:39.633313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcce08c-e717-4012-ba2b-46f69a8058df None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:39.639102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:39.639717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:39.640006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:39.640277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:39.642198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:39.654378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:39.655973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1dc34-a4c9-41bf-8130-438a68a9d0f0 service placement] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:39.666385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1125/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:43:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:43:49.661716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=32 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:43:56.676866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:43:56.678637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:43:56.690382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5758a3-812b-4edb-a405-59f54217470f service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:43:56.694974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:43:56.695584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:43:56.695859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:43:56.696130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:43:56.705541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:43:56.707020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:43:56.716639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1135/2260] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:43:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:44:06.712366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=6 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:44:15.570772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05151a91-132f-4208-b0ad-482ad07601ca None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:15.572746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05151a91-132f-4208-b0ad-482ad07601ca None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:15.573225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05151a91-132f-4208-b0ad-482ad07601ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:15.596749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-05151a91-132f-4208-b0ad-482ad07601ca None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:15.598379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-05151a91-132f-4208-b0ad-482ad07601ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:15.660532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1126/2261] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:44:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:44:15.710481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:44:15.711941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:15.796797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6ec30-3f00-4197-bbf9-ae7d6a3a9d15 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:44:15.801711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999bcce-b40f-4a26-a98d-a27e94750e98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:44:15.802357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999bcce-b40f-4a26-a98d-a27e94750e98 service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:15.802752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999bcce-b40f-4a26-a98d-a27e94750e98 service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:15.803033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999bcce-b40f-4a26-a98d-a27e94750e98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:15.811442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999bcce-b40f-4a26-a98d-a27e94750e98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:44:15.813072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999bcce-b40f-4a26-a98d-a27e94750e98 service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:44:15.821273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1136/2262] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:44:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:44:16.844676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01c42d-405e-4524-a78f-82eadf311e96 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:16.845244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01c42d-405e-4524-a78f-82eadf311e96 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:16.845244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01c42d-405e-4524-a78f-82eadf311e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:16.867935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7f01c42d-405e-4524-a78f-82eadf311e96 None None] MFA Rules not processed for user `43232b9b191047f9a3e6ebab7ad36f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:16.869456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7f01c42d-405e-4524-a78f-82eadf311e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:16.937969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1127/2263] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:44:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:44:25.818129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=32 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:44:26.929633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=50 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:44:29.959385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae72b-9643-4807-921e-562e07939411 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:29.959944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae72b-9643-4807-921e-562e07939411 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:29.959944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae72b-9643-4807-921e-562e07939411 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:29.961385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1137/2264] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:44:29.967399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394de55-0436-44f4-9234-a54514450f65 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:29.967877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394de55-0436-44f4-9234-a54514450f65 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:29.967992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394de55-0436-44f4-9234-a54514450f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:29.988943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-9394de55-0436-44f4-9234-a54514450f65 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:29.990209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9394de55-0436-44f4-9234-a54514450f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:30.000564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1128/2265] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:44:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:44:30.012339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae72b-9643-4807-921e-562e07939411 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:44:30.014219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2ceae72b-9643-4807-921e-562e07939411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:30.082434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae72b-9643-4807-921e-562e07939411 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:44:30.085982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7123f-1e73-4d0d-b868-8afd5e353cfe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:44:30.086640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7123f-1e73-4d0d-b868-8afd5e353cfe service glance] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.086915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7123f-1e73-4d0d-b868-8afd5e353cfe service glance] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.087186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7123f-1e73-4d0d-b868-8afd5e353cfe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.097161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7123f-1e73-4d0d-b868-8afd5e353cfe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:44:30.098648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7123f-1e73-4d0d-b868-8afd5e353cfe service glance] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:44:30.109681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1138/2266] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:44:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:44:30.184123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a6996-25f9-43a5-9665-818d6cdb3859 None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.184422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a6996-25f9-43a5-9665-818d6cdb3859 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.184689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a6996-25f9-43a5-9665-818d6cdb3859 None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.186307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1129/2267] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:44:30.190851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53ef7c-d357-471b-aa69-bc7d2a46e582 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.191139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53ef7c-d357-471b-aa69-bc7d2a46e582 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.191418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53ef7c-d357-471b-aa69-bc7d2a46e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.210834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ae53ef7c-d357-471b-aa69-bc7d2a46e582 None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:30.212064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ae53ef7c-d357-471b-aa69-bc7d2a46e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:30.218672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1139/2268] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:44:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:44:30.230690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a6996-25f9-43a5-9665-818d6cdb3859 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:44:30.243725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a6996-25f9-43a5-9665-818d6cdb3859 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:44:30.247687 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48aeaee-7ffd-4868-bcec-825b0724380f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:44:30.248291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aeaee-7ffd-4868-bcec-825b0724380f service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.248570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aeaee-7ffd-4868-bcec-825b0724380f service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.248838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aeaee-7ffd-4868-bcec-825b0724380f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.256560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48aeaee-7ffd-4868-bcec-825b0724380f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:44:30.257990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48aeaee-7ffd-4868-bcec-825b0724380f service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:44:30.265664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1130/2269] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:44:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:44:30.356371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c2383-a463-451c-a90d-6cbe2d046ac8 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.356659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c2383-a463-451c-a90d-6cbe2d046ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.356928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c2383-a463-451c-a90d-6cbe2d046ac8 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.358705 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1140/2270] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:44:30.364454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57897bef-fb19-490c-bcb5-ac6ed2e4bf66 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.364820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57897bef-fb19-490c-bcb5-ac6ed2e4bf66 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.365162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57897bef-fb19-490c-bcb5-ac6ed2e4bf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.385132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-57897bef-fb19-490c-bcb5-ac6ed2e4bf66 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:30.386360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-57897bef-fb19-490c-bcb5-ac6ed2e4bf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:30.397232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1131/2271] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:44:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:44:30.408399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c2383-a463-451c-a90d-6cbe2d046ac8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:44:30.421593 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c2383-a463-451c-a90d-6cbe2d046ac8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:44:30.426088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:44:30.426764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:30.427188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:30.427471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:30.438680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:44:30.440264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:44:30.451895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1141/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:44:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:44:36.086059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934176-8e38-471f-b910-ad1ece6d9d4d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:36.086570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934176-8e38-471f-b910-ad1ece6d9d4d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:36.086666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934176-8e38-471f-b910-ad1ece6d9d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:36.106189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0b934176-8e38-471f-b910-ad1ece6d9d4d None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:36.107415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0b934176-8e38-471f-b910-ad1ece6d9d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:36.115392 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1132/2273] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:44:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:44:36.127927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:44:36.139151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176a19-191c-4110-8963-db4c1664f618 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:44:36.144917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770ab6b-2398-43b8-8503-a425e6dd4677 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:44:36.145562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770ab6b-2398-43b8-8503-a425e6dd4677 service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:36.145847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770ab6b-2398-43b8-8503-a425e6dd4677 service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:36.146124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770ab6b-2398-43b8-8503-a425e6dd4677 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:36.154940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770ab6b-2398-43b8-8503-a425e6dd4677 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:44:36.156684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770ab6b-2398-43b8-8503-a425e6dd4677 service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:44:36.165129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1142/2274] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:44:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:44:46.101177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=15 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:44:46.161681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=41 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:44:52.669131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:44:52.669700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:44:52.669700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:44:52.690059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] MFA Rules not processed for user `192a8abd4c7a4535a42bea7648535dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:44:52.691299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:44:52.750201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1133/2275] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:44:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:45:02.697696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f010e-cb6e-463d-9712-6734b483579e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:02.698225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f010e-cb6e-463d-9712-6734b483579e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:02.698325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f010e-cb6e-463d-9712-6734b483579e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:02.718480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b66f010e-cb6e-463d-9712-6734b483579e None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:45:02.719701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b66f010e-cb6e-463d-9712-6734b483579e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:02.728303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1143/2276] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 9 02:45:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:45:02.742793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=25 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:02.744206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:02.756571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641206-eacb-4504-b645-9acc01c13ac5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:02.762101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:02.762814 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:02.763128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:02.763405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:02.771928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:02.773530 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:02.781950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1134/2277] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:45:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:45:12.713143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:12.778592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=6 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:13.004993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:13.005278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:13.005552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:13.025212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:45:13.026464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:13.034575 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1144/2278] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 9 02:45:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:45:13.049851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:13.061144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909abb5-d7da-40e8-a060-215abbd6e11b service swift] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:13.066863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:13.067464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:13.067763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:13.068034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:13.076458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:13.078240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:13.086114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1135/2279] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:45:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:45:19.726939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:19.729860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:19.812862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296a9e-b8c4-489c-a3a3-1d4418bd1876 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:19.820857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:19.822470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:19.822820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:19.823134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] PATH_INFO: `/v3/users/f6d6b64eb434414ab61f19b170385c3d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:19.824772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d6b64eb434414ab61f19b170385c3d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:19.827076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:19.911537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d6b64eb434414ab61f19b170385c3d'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:19.925896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d6b64eb434414ab61f19b170385c3d'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:19.926826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d6b64eb434414ab61f19b170385c3d'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:19.938634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d6b64eb434414ab61f19b170385c3d'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:19.954869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d6b64eb434414ab61f19b170385c3d'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:19.964879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1145/2280] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:45:19 2022] DELETE /identity/v3/users/f6d6b64eb434414ab61f19b170385c3d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:45:20.134982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:20.153581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d184a7-b6fd-4e06-a994-4028f6dfae66 service swift] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:20.217723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:20.218395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:20.218674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:20.218976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] PATH_INFO: `/v3/projects/f0a5508bdf5e4577b49783c3e424f1ee` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:20.220144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a5508bdf5e4577b49783c3e424f1ee)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:20.222029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:20.270250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a5508bdf5e4577b49783c3e424f1ee', 'request_id': 'req-35dc62f0-c9da-448f-b4a2-00cea1070a38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.277133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a5508bdf5e4577b49783c3e424f1ee', 'request_id': 'req-35dc62f0-c9da-448f-b4a2-00cea1070a38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.282902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a5508bdf5e4577b49783c3e424f1ee', 'request_id': 'req-35dc62f0-c9da-448f-b4a2-00cea1070a38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.296494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] The token cache is being invalidate because project f0a5508bdf5e4577b49783c3e424f1ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:45:20.296787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.297430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.299005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1136/2281] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:45:20 2022] DELETE /identity/v3/projects/f0a5508bdf5e4577b49783c3e424f1ee => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:45:20.303128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:20.305417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:20.318827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49055977-b2e3-491a-b4ea-8d05fec3ed4c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:20.385356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:20.385975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:20.386335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:20.386606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] PATH_INFO: `/v3/users/22f889d9fd8f4ce9883ec29e5951fb67` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:20.387740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f889d9fd8f4ce9883ec29e5951fb67)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:20.390625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:20.406430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:20.454332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f889d9fd8f4ce9883ec29e5951fb67'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.463640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f889d9fd8f4ce9883ec29e5951fb67'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.464315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f889d9fd8f4ce9883ec29e5951fb67'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.473636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f889d9fd8f4ce9883ec29e5951fb67'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.487198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-36dc4b65-480f-4116-b3c3-97c6a8655304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f889d9fd8f4ce9883ec29e5951fb67'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.496601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1146/2282] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:45:20 2022] DELETE /identity/v3/users/22f889d9fd8f4ce9883ec29e5951fb67 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:45:20.695789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:20.697315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:20.734310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc62f0-c9da-448f-b4a2-00cea1070a38 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:20.864225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:20.864897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:20.865176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:20.865450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] PATH_INFO: `/v3/projects/5099eb118dd4449a9dea3ece36073723` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:20.866671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5099eb118dd4449a9dea3ece36073723)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:20.868635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:20.898271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:20.936299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5099eb118dd4449a9dea3ece36073723', 'request_id': 'req-2213069f-6709-4d9b-9eb0-009808f35b34'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.943825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5099eb118dd4449a9dea3ece36073723', 'request_id': 'req-2213069f-6709-4d9b-9eb0-009808f35b34'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.952517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5099eb118dd4449a9dea3ece36073723', 'request_id': 'req-2213069f-6709-4d9b-9eb0-009808f35b34'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.968802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] The token cache is being invalidate because project 5099eb118dd4449a9dea3ece36073723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:45:20.969079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.969896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:45:20.971495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1137/2283] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:45:20 2022] DELETE /identity/v3/projects/5099eb118dd4449a9dea3ece36073723 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:45:23.845065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:23.845618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:23.845618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:23.867981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] MFA Rules not processed for user `2aaa8a7f386e4ccc9ab5c356cdf1704f`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:45:23.869520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:23.925951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1147/2284] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:45:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:45:25.891396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:25.892926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:25.951399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213069f-6709-4d9b-9eb0-009808f35b34 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:25.955474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:25.956114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:25.956382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:25.956785 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:25.958669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:26.035451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:26.036938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:26.047731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1138/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:28.436698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:28.448481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091f0fb-5127-4bde-b788-8389d0c18ec1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:28.452297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:28.452922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:28.453204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:28.453481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:28.455317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:28.469648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:28.470694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d824c3-cf43-4ab8-8c82-c5e38b4afc07 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:28.481903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1148/2286] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:36.042974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:36.443202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:36.445249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:36.459838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1388841-952c-48c3-ad95-6597811f3879 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:36.467323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:36.468076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:36.468567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:36.468910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:36.471039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:36.530287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:36.531994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:36.540967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1139/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:38.476964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=52,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:39.051324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:39.051611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:39.051884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:39.071097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] MFA Rules not processed for user `43232b9b191047f9a3e6ebab7ad36f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:45:39.072274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:39.079476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1149/2288] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:45:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:45:39.452899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:39.462724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb5ae1-0c9a-444d-a206-a7d6c52c6c6b service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:39.466678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:39.467285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:39.467565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:39.467837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:39.470114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:39.481703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:39.483271 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:39.491934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1140/2289] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:40.027658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:40.029083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:40.103489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d4c0-1208-4398-8fc1-6e948c06eda0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:40.108770 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:40.109393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:40.109700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:40.110385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:40.112215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:40.129082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:40.131072 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:40.142245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1150/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:43.692726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:43.694466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:43.705201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f463f6-db30-417b-8fc4-f3a49edd4f93 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:43.708989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:43.709593 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:43.709885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:43.710171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:43.711940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:43.721349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:43.722829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:43.730743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1141/2291] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:50.137551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=35 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:53.450781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:53.463094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3fb0a-1b53-4a73-b4b3-9c3f64998b00 service placement] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:53.468597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:53.469186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:53.469665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:53.469937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:53.471896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:53.533781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:53.535748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc8471-c0ea-4a52-9d41-8dc3b3b6ce2b service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:53.544948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1151/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:53.727227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=40 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:45:56.175375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:45:56.177612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:56.197216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2fe9f-952b-4394-8997-7d603a56d586 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:45:56.203203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:45:56.203962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:56.204450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:56.204869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:56.206857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:56.219681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:45:56.228061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:45:56.240436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1142/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:45:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:45:58.575474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfa807-582c-40b9-8013-00e10cd39004 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:45:58.576482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfa807-582c-40b9-8013-00e10cd39004 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:45:58.577024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfa807-582c-40b9-8013-00e10cd39004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:45:58.600160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-dbdfa807-582c-40b9-8013-00e10cd39004 None None] MFA Rules not processed for user `81447e83a97b4519ac6d0eac9a06bdef`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:45:58.601747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-dbdfa807-582c-40b9-8013-00e10cd39004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:45:58.661355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1152/2294] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:45:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:01.363417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:01.377877 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126fa6d-df0b-4eca-a601-78f6d9d5ebd8 service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:01.384074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:01.385127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:01.385406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:01.385671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:01.387470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:01.400778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:01.402458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:01.423227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1143/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:01.697921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742df2b-40ab-4df4-a011-372636943336 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:01.698480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742df2b-40ab-4df4-a011-372636943336 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:01.698564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742df2b-40ab-4df4-a011-372636943336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:01.719310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1742df2b-40ab-4df4-a011-372636943336 None None] MFA Rules not processed for user `90fa94f90d49485a81d502929bcd0b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:01.720716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1742df2b-40ab-4df4-a011-372636943336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:01.780784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1153/2296] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:02.303085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:02.313087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a153a92-18a8-417f-a933-f5ed9e4f1155 service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:02.317056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:02.317752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:02.318098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:02.318458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:02.320339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:02.330425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:02.331955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:02.340526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1144/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:02.653223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742df2b-40ab-4df4-a011-372636943336 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:02.664558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742df2b-40ab-4df4-a011-372636943336 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:02.669171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:02.669806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:02.670085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:02.670373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:02.672600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:02.724040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:02.743642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:02.745122 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:02.757214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1154/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:06.895461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:06.897251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:06.913069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1086b-8eb9-4e03-aa30-e32a9fa22bde service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:06.919492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:06.920591 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:06.920986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:06.921356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] PATH_INFO: `/v3/users/192a8abd4c7a4535a42bea7648535dc3` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:06.922851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=192a8abd4c7a4535a42bea7648535dc3)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:06.925062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:06.986966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192a8abd4c7a4535a42bea7648535dc3'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:06.988489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192a8abd4c7a4535a42bea7648535dc3'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.000016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192a8abd4c7a4535a42bea7648535dc3'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.008142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192a8abd4c7a4535a42bea7648535dc3'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.021595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-34fcc028-66ab-49ac-a210-9567a2aff27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192a8abd4c7a4535a42bea7648535dc3'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.028170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1145/2299] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:06 2022] DELETE /identity/v3/users/192a8abd4c7a4535a42bea7648535dc3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:07.200115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.211390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570661e-f20c-4f28-87d3-cbee04255c37 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.288367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.289114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.289441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.289796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] PATH_INFO: `/v3/projects/65aab6ab3ce94cbd93916edf9c3eedc0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.291023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65aab6ab3ce94cbd93916edf9c3eedc0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.293043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.344930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65aab6ab3ce94cbd93916edf9c3eedc0', 'request_id': 'req-3b1a6300-d43b-45a2-9204-a89401a763a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.353588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65aab6ab3ce94cbd93916edf9c3eedc0', 'request_id': 'req-3b1a6300-d43b-45a2-9204-a89401a763a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.360382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65aab6ab3ce94cbd93916edf9c3eedc0', 'request_id': 'req-3b1a6300-d43b-45a2-9204-a89401a763a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.371829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] The token cache is being invalidate because project 65aab6ab3ce94cbd93916edf9c3eedc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:07.372263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.373047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3b1a6300-d43b-45a2-9204-a89401a763a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:07.374951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1155/2300] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:07 2022] DELETE /identity/v3/projects/65aab6ab3ce94cbd93916edf9c3eedc0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:07.380006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.380345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.380752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.402717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:07.404855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:07.470962 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1146/2301] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:07.480908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.481174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.481980 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.504828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:07.506352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:07.517099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1156/2302] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:07.521309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.531889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe908a-a677-4fea-a051-73dca043c20b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.538314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.539068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.539354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.539682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.540356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:46:07.540671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.542499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.555580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1147/2303] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:46:07.559868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.570812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8dcbc-e558-48ac-a360-035821d6c163 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.577067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.577694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.577976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.578271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.578915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.580906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.613514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1157/2304] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:07 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:46:07.614787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.625089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900357-aef1-4864-9938-5b158b10f96c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.646222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:07.691114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.691649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.692080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.692365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.693008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.694995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.724788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1148/2305] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:07 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:46:07.726438 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.738219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e43961-e245-4ebe-bcac-00143eb91469 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.744195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.745837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.746237 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.746595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.747333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.749294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.765498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1158/2306] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:07.769242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.779752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988c1f5-0930-43ab-95b2-c668ec3da49c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.785549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb6316-63b3-47c9-b6e4-01fc586168d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.786210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6316-63b3-47c9-b6e4-01fc586168d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.786501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6316-63b3-47c9-b6e4-01fc586168d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.786779 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6316-63b3-47c9-b6e4-01fc586168d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.787448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb6316-63b3-47c9-b6e4-01fc586168d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.789076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb6316-63b3-47c9-b6e4-01fc586168d6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.811581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1149/2307] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:07.812710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:07.825221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6bd23-8326-4f48-88ad-21cbad1ca4b3 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:07.830808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:07.831649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.832001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.832344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] PATH_INFO: `/v3/projects/9353662f7d4d4457a7a8797e8915ac80/users/35990015a20d4b7caaccc9ad123dd1b0/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.848485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9353662f7d4d4457a7a8797e8915ac80, user_id=35990015a20d4b7caaccc9ad123dd1b0, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:07.851423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:07.869903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1159/2308] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:07 2022] PUT /identity/v3/projects/9353662f7d4d4457a7a8797e8915ac80/users/35990015a20d4b7caaccc9ad123dd1b0/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:07.880223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:07.880514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:07.880796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:07.912542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] MFA Rules not processed for user `35990015a20d4b7caaccc9ad123dd1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:07.914707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:08.019022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1150/2309] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:08.023609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.038613 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:08.091704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c97909-7e23-4b32-b2f4-bd4152aeee58 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.097115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.097753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.098020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.098326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.098961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.100931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.129068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1160/2310] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:08 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:08.133708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.145654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67862cd-e52a-4127-ac07-3941ed4f3b93 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.204430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.204945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.205224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.205493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.206186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.208175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.230029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1151/2311] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:08 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:08.234313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.245306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56320238-374f-4595-91b2-4de871b46316 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.250558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.251162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.251442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.251714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.252368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.254193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.264974 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1161/2312] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:08.269426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.280187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de8450-04c6-4be6-8d2c-d8cdce311489 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.286615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.287274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.287555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.287830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] PATH_INFO: `/v3/projects/6f3e4d28bde7481689e9da48ca9b9127/users/3b7d7ff51e7a4319a45dc25008789697/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.303494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e4d28bde7481689e9da48ca9b9127, user_id=3b7d7ff51e7a4319a45dc25008789697, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.306181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.308377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:08.324968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1152/2313] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:08 2022] PUT /identity/v3/projects/6f3e4d28bde7481689e9da48ca9b9127/users/3b7d7ff51e7a4319a45dc25008789697/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:08.329071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.341975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc26b-ce65-4653-8d0a-4a46f62dbf6e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.403511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e308c-0210-498a-a7ab-1248cb258120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.404373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e308c-0210-498a-a7ab-1248cb258120 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.404970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e308c-0210-498a-a7ab-1248cb258120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.405450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e308c-0210-498a-a7ab-1248cb258120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.406362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e308c-0210-498a-a7ab-1248cb258120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.408266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e308c-0210-498a-a7ab-1248cb258120 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.420730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1162/2314] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:08.425147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.439136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bcc1c-6785-4a5b-8af7-7a51234a4a0f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.445441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.446112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.446428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.446701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] PATH_INFO: `/v3/projects/6f3e4d28bde7481689e9da48ca9b9127/users/3b7d7ff51e7a4319a45dc25008789697/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.448750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e4d28bde7481689e9da48ca9b9127, user_id=3b7d7ff51e7a4319a45dc25008789697, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.451520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.470272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1153/2315] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:08 2022] PUT /identity/v3/projects/6f3e4d28bde7481689e9da48ca9b9127/users/3b7d7ff51e7a4319a45dc25008789697/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:08.475120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.475414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.475695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.503688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] MFA Rules not processed for user `3b7d7ff51e7a4319a45dc25008789697`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:08.504933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:08.578658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1163/2316] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:08.590714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:08.592727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:08.661038 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9131b-312a-4f3a-b23a-d9f35bc3b36e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:08.665340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5e7b1-c107-42af-aca6-cdb0b6a51e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:08.665967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e7b1-c107-42af-aca6-cdb0b6a51e95 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:08.666464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e7b1-c107-42af-aca6-cdb0b6a51e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:08.666741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e7b1-c107-42af-aca6-cdb0b6a51e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:08.676270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5e7b1-c107-42af-aca6-cdb0b6a51e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:08.677716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5e7b1-c107-42af-aca6-cdb0b6a51e95 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:08.687585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1154/2317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:09.158682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:09.164259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:09.225236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a0247-5b51-4246-b009-75858bcbd917 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:09.229956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:09.230547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:09.230825 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:09.231095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] PATH_INFO: `/v3/users/35990015a20d4b7caaccc9ad123dd1b0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:09.232214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35990015a20d4b7caaccc9ad123dd1b0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:09.234185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:09.286269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35990015a20d4b7caaccc9ad123dd1b0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.296489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35990015a20d4b7caaccc9ad123dd1b0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.298267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35990015a20d4b7caaccc9ad123dd1b0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.312330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35990015a20d4b7caaccc9ad123dd1b0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.331944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35990015a20d4b7caaccc9ad123dd1b0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.339957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1164/2318] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:09 2022] DELETE /identity/v3/users/35990015a20d4b7caaccc9ad123dd1b0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:09.346069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:09.346574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:09.346692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:09.367963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:09.369193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:09.429950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1155/2319] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:09.439962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:09.441468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:09.507706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727bc24-f988-4e1c-89f7-93ad1990dbf9 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:09.512423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:09.513047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:09.513325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:09.513596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:09.523711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:09.525195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:09.534971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1165/2320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:09.829350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:09.831398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:09.847915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079b52c-7c16-4f94-af2b-3d6663763d47 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:09.853320 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:09.854220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:09.854501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:09.854863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] PATH_INFO: `/v3/projects/9353662f7d4d4457a7a8797e8915ac80` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:09.855989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9353662f7d4d4457a7a8797e8915ac80)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:09.857912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:09.895342 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:09.896906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:09.926936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9353662f7d4d4457a7a8797e8915ac80', 'request_id': 'req-f5015097-2a82-4f67-960b-468d326342f9'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.934408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9353662f7d4d4457a7a8797e8915ac80', 'request_id': 'req-f5015097-2a82-4f67-960b-468d326342f9'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.935878 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:09.941393 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9353662f7d4d4457a7a8797e8915ac80', 'request_id': 'req-f5015097-2a82-4f67-960b-468d326342f9'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.956127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] The token cache is being invalidate because project 9353662f7d4d4457a7a8797e8915ac80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:09.956452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.957074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:09.958692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1156/2321] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:09 2022] DELETE /identity/v3/projects/9353662f7d4d4457a7a8797e8915ac80 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:09.963229 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:09.964749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:09.975953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:09.986427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe24686-b51f-4a9a-9fe1-4ddb3fd617ee service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:10.038581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015097-2a82-4f67-960b-468d326342f9 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:10.044101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:10.044747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.045066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.045343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] PATH_INFO: `/v3/users/3b7d7ff51e7a4319a45dc25008789697` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.047228 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7d7ff51e7a4319a45dc25008789697)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:10.049309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:10.056412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:10.057132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.057571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.057912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.059726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:10.074192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:10.075328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:10.087020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1166/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:46:10.110083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7d7ff51e7a4319a45dc25008789697'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.111472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7d7ff51e7a4319a45dc25008789697'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.124453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7d7ff51e7a4319a45dc25008789697'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.132286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7d7ff51e7a4319a45dc25008789697'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.144057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-01a4c4a1-1c1b-48d6-9d83-dd72c51c7969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7d7ff51e7a4319a45dc25008789697'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.149999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1157/2323] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:09 2022] DELETE /identity/v3/users/3b7d7ff51e7a4319a45dc25008789697 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:46:10.511482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:10.525278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91381422-bdb4-4490-bb2c-546d5a7c5a02 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:10.587189 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:10.587835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.588115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.588386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] PATH_INFO: `/v3/projects/6f3e4d28bde7481689e9da48ca9b9127` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.589717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3e4d28bde7481689e9da48ca9b9127)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:10.591706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:10.641181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3e4d28bde7481689e9da48ca9b9127', 'request_id': 'req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.643752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:10.649676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3e4d28bde7481689e9da48ca9b9127', 'request_id': 'req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.656143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3e4d28bde7481689e9da48ca9b9127', 'request_id': 'req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.668232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] The token cache is being invalidate because project 6f3e4d28bde7481689e9da48ca9b9127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:10.668522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.669238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d48e3ea9-6ac1-4e6a-a21c-4536b9e27938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:10.672921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1167/2324] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:10 2022] DELETE /identity/v3/projects/6f3e4d28bde7481689e9da48ca9b9127 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:10.678644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.679123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.679220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.699388 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:10.700598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:10.764691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1158/2325] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:10.772839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.773125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.773410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.794439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:10.795659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:10.805111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1168/2326] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:10.809983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:10.821953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08157bc-60df-41f1-ae40-e04cebc2568e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:10.827617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:10.828217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.828500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.828775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.829420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:46:10.829722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:10.831359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:10.843816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1159/2327] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:46:10.848230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:10.857786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2345aac-87c2-46bb-b81b-c9044508a5cd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:10.863571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:10.864166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.864479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.864743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.865369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:10.867328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:10.898143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1169/2328] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:10 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:10.902665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:10.913663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13233b5-abde-406e-95a0-fabbd7b83472 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:10.922738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:10.973571 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:10.974187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:10.974488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:10.974762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:10.975387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:10.977344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.002533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1160/2329] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:10 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:11.009249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.026066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a846ec-5609-4215-bdd8-d8f4cf63f229 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.034733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.035339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.035619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.035888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.036541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.038179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.048620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1170/2330] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:11.052957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.063354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2b077-7100-4b43-8d3d-6b777a94f149 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.068281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22273a-d2e4-4912-9d79-e96c3d600292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.068882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22273a-d2e4-4912-9d79-e96c3d600292 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.069148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22273a-d2e4-4912-9d79-e96c3d600292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.069423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22273a-d2e4-4912-9d79-e96c3d600292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.070069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22273a-d2e4-4912-9d79-e96c3d600292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.071706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22273a-d2e4-4912-9d79-e96c3d600292 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.082652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1161/2331] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:11.086919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.097504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f87cd6-bc7e-4e3f-b7d2-3fa752b62928 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.102497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.103141 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.103422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.103697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] PATH_INFO: `/v3/projects/db20d6c419ed45cbb063cdae11adb5b2/users/84d96ccb93b24f168f981df848f8a248/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.118735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db20d6c419ed45cbb063cdae11adb5b2, user_id=84d96ccb93b24f168f981df848f8a248, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.121386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.137158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1171/2332] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:11 2022] PUT /identity/v3/projects/db20d6c419ed45cbb063cdae11adb5b2/users/84d96ccb93b24f168f981df848f8a248/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:11.147792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.148065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.148334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.175600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] MFA Rules not processed for user `84d96ccb93b24f168f981df848f8a248`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:11.176810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:11.241906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1162/2333] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:11.246494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.309715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e29c85-4262-464d-921b-c14217a8f37a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.314389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.315003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.315285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.315557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.316232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.318146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.351400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1172/2334] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:11 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:11.352589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.364272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a22e9-4ba0-4256-8378-981db7b13a2c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.431871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.432517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.432796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.433073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.433701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.435647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.461930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1163/2335] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:11 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:11.463194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.473893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf0f7e-8ca4-4ba1-8ffc-06fbef72af39 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.479820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.480437 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.480726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.480997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.481652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.483294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.494662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1173/2336] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:11.498729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.509351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210855-8223-4ec6-9f89-2907a760ff6a admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.514950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.515627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.515905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.516174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] PATH_INFO: `/v3/projects/b6d78b0b67134f909200723cefdec1e4/users/5c6dc973647c49b58b2d92794e9ff75a/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.527049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:11.536080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d78b0b67134f909200723cefdec1e4, user_id=5c6dc973647c49b58b2d92794e9ff75a, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.538944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.555777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1164/2337] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:11 2022] PUT /identity/v3/projects/b6d78b0b67134f909200723cefdec1e4/users/5c6dc973647c49b58b2d92794e9ff75a/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:11.559753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.570604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0c254-d556-4680-9a1e-297156844c21 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.618606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:11.629029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e13db0-b105-4dd5-87ea-3e4b34609354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.629771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e13db0-b105-4dd5-87ea-3e4b34609354 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.630109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e13db0-b105-4dd5-87ea-3e4b34609354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.630595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e13db0-b105-4dd5-87ea-3e4b34609354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.631249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e13db0-b105-4dd5-87ea-3e4b34609354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.632849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e13db0-b105-4dd5-87ea-3e4b34609354 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.642835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1174/2338] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:11.647343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.659493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e078083-40bc-420d-8e7d-e8d64dcef3bb admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.665372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.666029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.666368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.666636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] PATH_INFO: `/v3/projects/b6d78b0b67134f909200723cefdec1e4/users/5c6dc973647c49b58b2d92794e9ff75a/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.668819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d78b0b67134f909200723cefdec1e4, user_id=5c6dc973647c49b58b2d92794e9ff75a, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.671564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.682809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1165/2339] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:11 2022] PUT /identity/v3/projects/b6d78b0b67134f909200723cefdec1e4/users/5c6dc973647c49b58b2d92794e9ff75a/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:11.694517 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.694812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.695085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.725043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] MFA Rules not processed for user `5c6dc973647c49b58b2d92794e9ff75a`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:11.726292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:11.796108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1175/2340] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:11.807952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:11.809847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:11.871463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259c368-730d-429b-9ee7-98bc13e4fcb6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:11.877299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895a243-98e9-4eb5-9795-e061908419ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:11.878016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895a243-98e9-4eb5-9795-e061908419ca service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:11.878339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895a243-98e9-4eb5-9795-e061908419ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:11.878608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895a243-98e9-4eb5-9795-e061908419ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:11.888055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895a243-98e9-4eb5-9795-e061908419ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:11.889482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895a243-98e9-4eb5-9795-e061908419ca service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:11.899048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1166/2341] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:11.966792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:12.043794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cc44f-e582-41d9-924d-3de639b19d93 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:12.050679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:12.051311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:12.051586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:12.051860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] PATH_INFO: `/v3/users/84d96ccb93b24f168f981df848f8a248` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:12.053166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d96ccb93b24f168f981df848f8a248)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:12.055184 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:12.105444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d96ccb93b24f168f981df848f8a248'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.114208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d96ccb93b24f168f981df848f8a248'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.114856 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d96ccb93b24f168f981df848f8a248'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.121987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d96ccb93b24f168f981df848f8a248'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.130128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:12.136146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d96ccb93b24f168f981df848f8a248'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.143746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1176/2342] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:11 2022] DELETE /identity/v3/users/84d96ccb93b24f168f981df848f8a248 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:12.148490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:12.148778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:12.149054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:12.168751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:12.170013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:12.235387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1167/2343] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:12.245996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:12.247553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:12.308392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887889d-9f82-4d4e-8a45-57f83395b6b5 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:12.312264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:12.312882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:12.313163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:12.313436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:12.322887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:12.324383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:12.334120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1177/2344] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:12.614529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:12.616142 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:12.633463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd40b0-7344-4adf-9daf-7128d78999a3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:12.638211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:12.638864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:12.639145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:12.639419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] PATH_INFO: `/v3/projects/db20d6c419ed45cbb063cdae11adb5b2` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:12.640541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db20d6c419ed45cbb063cdae11adb5b2)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:12.643346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:12.693751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db20d6c419ed45cbb063cdae11adb5b2', 'request_id': 'req-e8345924-e03d-4d40-82ad-49608e6747f0'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.700876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db20d6c419ed45cbb063cdae11adb5b2', 'request_id': 'req-e8345924-e03d-4d40-82ad-49608e6747f0'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.710092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db20d6c419ed45cbb063cdae11adb5b2', 'request_id': 'req-e8345924-e03d-4d40-82ad-49608e6747f0'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.722826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:12.729021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] The token cache is being invalidate because project db20d6c419ed45cbb063cdae11adb5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:12.729803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.730467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.732939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1168/2345] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:12 2022] DELETE /identity/v3/projects/db20d6c419ed45cbb063cdae11adb5b2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:12.737176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:12.747248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2773-eba3-40ac-ad41-d77196e074b1 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:12.811940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:12.812779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:12.813107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:12.813454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] PATH_INFO: `/v3/users/5c6dc973647c49b58b2d92794e9ff75a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:12.814728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c6dc973647c49b58b2d92794e9ff75a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:12.816713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:12.863801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c6dc973647c49b58b2d92794e9ff75a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.872503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c6dc973647c49b58b2d92794e9ff75a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.873278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c6dc973647c49b58b2d92794e9ff75a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.880293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c6dc973647c49b58b2d92794e9ff75a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.891591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-3fd5103e-adc5-438d-a136-4ce6aafb2b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c6dc973647c49b58b2d92794e9ff75a'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:12.897644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1178/2346] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:12 2022] DELETE /identity/v3/users/5c6dc973647c49b58b2d92794e9ff75a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:13.168405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.170193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:13.183931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8345924-e03d-4d40-82ad-49608e6747f0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.247210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.247855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.248135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.248411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] PATH_INFO: `/v3/projects/b6d78b0b67134f909200723cefdec1e4` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.249540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d78b0b67134f909200723cefdec1e4)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.251733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.301202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d78b0b67134f909200723cefdec1e4', 'request_id': 'req-f2edbbab-9b75-4373-86ef-e2655cdc6640'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:13.307324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d78b0b67134f909200723cefdec1e4', 'request_id': 'req-f2edbbab-9b75-4373-86ef-e2655cdc6640'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:13.312940 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d78b0b67134f909200723cefdec1e4', 'request_id': 'req-f2edbbab-9b75-4373-86ef-e2655cdc6640'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:13.327934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] The token cache is being invalidate because project b6d78b0b67134f909200723cefdec1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:13.328420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:13.329058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f2edbbab-9b75-4373-86ef-e2655cdc6640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:13.330726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1169/2347] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:13 2022] DELETE /identity/v3/projects/b6d78b0b67134f909200723cefdec1e4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:13.335570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.335862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.336139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.355350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:13.356577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:13.366388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:13.442288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1179/2348] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:13.444165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.444520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.444862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.464764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:13.466140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:13.475392 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1170/2349] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:13.479794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.490143 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ff87-b7fb-4db9-a54d-2999ffc8c5dd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.494898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.495508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.495794 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.496060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.496753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:46:13.497058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.498702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.510511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1180/2350] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:46:13.514835 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.525184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940ba9-d416-4139-add1-d61a1b8b39ad None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.531347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.532045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.532394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.532674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.533314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.535359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.569469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1171/2351] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:13 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:13.570714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.581845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ab53f-1890-4b70-b5bd-fd5fdf23630f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.651763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.652473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.652852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.653235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.653954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.656257 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.686723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1181/2352] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:13 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:13.688288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.699323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78dcc7-bc23-40c6-a98f-95e30c9c8596 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.704576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.705296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.705727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.706069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.706831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.708497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.719089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1172/2353] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:13.723216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.733483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495b47c-fd2d-474c-aac4-19acbf77ca9c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.738649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efda9b8-69b6-4ed5-937c-79db0bd00842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.739296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efda9b8-69b6-4ed5-937c-79db0bd00842 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.739524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efda9b8-69b6-4ed5-937c-79db0bd00842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.739795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efda9b8-69b6-4ed5-937c-79db0bd00842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.740452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efda9b8-69b6-4ed5-937c-79db0bd00842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.742088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efda9b8-69b6-4ed5-937c-79db0bd00842 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.753068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1182/2354] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:13.757104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.767622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4cdd3-93c1-4577-a65d-f8a3a15a5da6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:13.772905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:13.773581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.773866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.774189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] PATH_INFO: `/v3/projects/c5ec1530e5e841ca812e0142a707a4f8/users/6360fd09d86f4d308dc414dcbcf9cb20/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.789507 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ec1530e5e841ca812e0142a707a4f8, user_id=6360fd09d86f4d308dc414dcbcf9cb20, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:13.792935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:13.821239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1173/2355] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:13 2022] PUT /identity/v3/projects/c5ec1530e5e841ca812e0142a707a4f8/users/6360fd09d86f4d308dc414dcbcf9cb20/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:13.824002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:13.824372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:13.824745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:13.856001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] MFA Rules not processed for user `6360fd09d86f4d308dc414dcbcf9cb20`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:13.857418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:13.930938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1183/2356] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:46:13.935172 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:13.967373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:14.003119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5471798-1581-44f8-9f9b-fd1b56e4dede admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.007239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.007857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.008136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.008410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.009035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.011054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.039448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1174/2357] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:13 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:14.043468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.054580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa472-e339-4f40-9270-920c849eccd8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.059374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:14.116139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.116756 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.117022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.117310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.117959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.119939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.140982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1184/2358] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:14 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:14.146022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.155934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3a631-89d4-4f98-8576-4a5e76f0d20d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.161401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.162008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.162324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.162572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.163238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.164995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.176649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1175/2359] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:14.180913 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.191287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46416340-a466-412e-9f30-47beb5802b19 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.196506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0846a29-0503-47f0-b13b-93a3cc4d7479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.197115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846a29-0503-47f0-b13b-93a3cc4d7479 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.197386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846a29-0503-47f0-b13b-93a3cc4d7479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.197656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846a29-0503-47f0-b13b-93a3cc4d7479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.198359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0846a29-0503-47f0-b13b-93a3cc4d7479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.200022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0846a29-0503-47f0-b13b-93a3cc4d7479 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.210400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1185/2360] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:14.214833 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.225801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387255d-161e-4a3e-ab74-26d6db9e3455 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.231413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.232073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.232390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.232665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] PATH_INFO: `/v3/projects/322b626dd96d4b0fa8cb85b5693d85c3/users/5b3ad557aa464597a4c5b3a8b2b48ac4/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.248313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322b626dd96d4b0fa8cb85b5693d85c3, user_id=5b3ad557aa464597a4c5b3a8b2b48ac4, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.251144 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.265898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1176/2361] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:14 2022] PUT /identity/v3/projects/322b626dd96d4b0fa8cb85b5693d85c3/users/5b3ad557aa464597a4c5b3a8b2b48ac4/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:14.276552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.276866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.277119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.305343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] MFA Rules not processed for user `5b3ad557aa464597a4c5b3a8b2b48ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:14.306709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:14.376545 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1186/2362] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:14.381597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.462200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26271a8d-156d-4837-8367-1af15c4cdb07 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.467089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.467967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.468201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.468555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.469253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.471313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.507511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1177/2363] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:14 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:14.511592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.524146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0106e-7b45-4271-bc16-895f46c898d1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.588326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.588990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.589275 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.589550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.590216 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.592201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.619566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1187/2364] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:14 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:14.623738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.635623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514405ba-6527-4a6b-899f-c6a1a5c26f6c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.641221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.641850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.642134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.642455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.643156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.644794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.660077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1178/2365] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:14.661640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.672753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11545dbd-8549-41a8-becb-afbb05cf0eac admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.678619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.679120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.679412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.679694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] PATH_INFO: `/v3/projects/5504a3307d41435aa42a7e0ac048993f/users/7cfb83465c9e4459bf0eff30d6e008f7/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.682635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:14.700187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5504a3307d41435aa42a7e0ac048993f, user_id=7cfb83465c9e4459bf0eff30d6e008f7, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.703241 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.719468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1188/2366] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:14 2022] PUT /identity/v3/projects/5504a3307d41435aa42a7e0ac048993f/users/7cfb83465c9e4459bf0eff30d6e008f7/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:14.724011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.735455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccef10-04ce-4ea6-b2f8-0dffd7f76810 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.787440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:14.801069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ab730-e708-40b6-a004-41ef1a41f708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.801718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ab730-e708-40b6-a004-41ef1a41f708 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.802045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ab730-e708-40b6-a004-41ef1a41f708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.802399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ab730-e708-40b6-a004-41ef1a41f708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.803299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ab730-e708-40b6-a004-41ef1a41f708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.805030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ab730-e708-40b6-a004-41ef1a41f708 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.819305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1179/2367] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:14.820502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.831841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3537e9-b18d-4c05-845f-f76f3fadccfb admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:14.836546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:14.837217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.837496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.837769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] PATH_INFO: `/v3/projects/5504a3307d41435aa42a7e0ac048993f/users/7cfb83465c9e4459bf0eff30d6e008f7/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.840079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5504a3307d41435aa42a7e0ac048993f, user_id=7cfb83465c9e4459bf0eff30d6e008f7, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:14.843100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:14.861352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1189/2368] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:14 2022] PUT /identity/v3/projects/5504a3307d41435aa42a7e0ac048993f/users/7cfb83465c9e4459bf0eff30d6e008f7/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:14.865345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:14.865598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:14.865879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:14.896336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] MFA Rules not processed for user `7cfb83465c9e4459bf0eff30d6e008f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:14.897617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:14.967744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1180/2369] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:14.978684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:14.980573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:15.039692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011b8ae-2627-49b8-bf9d-344f7b8bf67f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:15.043670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:15.044289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:15.044564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:15.044833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:15.054458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:15.055901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:15.065331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1190/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:15.596807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:15.598449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:15.611419 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45cda3-9cba-4c95-8901-4b34ffad0491 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:15.615336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:15.615949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:15.616234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:15.616496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:15.678120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:15.679740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:15.689334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1181/2371] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:16.744363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:16.747107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:16.825066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1369f4-c533-4219-9736-9b697c683e48 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:16.832111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:16.832973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:16.833250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:16.833516 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:16.835708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:16.897060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:16.898787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:16.908220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1191/2372] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:19.798803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:19.808015 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677e615-9512-4c28-bc2c-2a4c3e3203d7 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:19.812467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147659a-fa2e-486f-b941-ccefcf74ba1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:19.813149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147659a-fa2e-486f-b941-ccefcf74ba1f service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:19.813496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147659a-fa2e-486f-b941-ccefcf74ba1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:19.813840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147659a-fa2e-486f-b941-ccefcf74ba1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:19.871405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147659a-fa2e-486f-b941-ccefcf74ba1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:19.872859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147659a-fa2e-486f-b941-ccefcf74ba1f service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:19.881748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1182/2373] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:22.059136 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:22.069885 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:22.123917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40fbbd-3404-49f1-bbe2-84dcd0423e13 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:22.129262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:22.129910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:22.130230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:22.130522 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] PATH_INFO: `/v3/users/6360fd09d86f4d308dc414dcbcf9cb20` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:22.131682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6360fd09d86f4d308dc414dcbcf9cb20)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:22.133624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:22.186106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6360fd09d86f4d308dc414dcbcf9cb20'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.194508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6360fd09d86f4d308dc414dcbcf9cb20'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.195144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6360fd09d86f4d308dc414dcbcf9cb20'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.202277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6360fd09d86f4d308dc414dcbcf9cb20'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.213692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6360fd09d86f4d308dc414dcbcf9cb20'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.219655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1192/2374] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:22 2022] DELETE /identity/v3/users/6360fd09d86f4d308dc414dcbcf9cb20 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:22.224481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae39c55-ed27-4763-850d-37170264408a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:22.225249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae39c55-ed27-4763-850d-37170264408a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:22.225249 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae39c55-ed27-4763-850d-37170264408a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:22.244218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-7ae39c55-ed27-4763-850d-37170264408a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:22.245434 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7ae39c55-ed27-4763-850d-37170264408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:22.249535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:22.305738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1183/2375] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:22.315764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:22.317502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:22.376911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cfe95-f015-41e9-ae62-1e7617af7e8c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:22.381218 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:22.381836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:22.382107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:22.382466 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:22.392968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:22.394590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:22.405110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1193/2376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:22.718669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae39c55-ed27-4763-850d-37170264408a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:22.720423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7ae39c55-ed27-4763-850d-37170264408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:22.733466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae39c55-ed27-4763-850d-37170264408a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:22.738747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:22.739404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:22.739804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:22.740031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] PATH_INFO: `/v3/projects/c5ec1530e5e841ca812e0142a707a4f8` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:22.741194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ec1530e5e841ca812e0142a707a4f8)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:22.743146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:22.799508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ec1530e5e841ca812e0142a707a4f8', 'request_id': 'req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.805850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ec1530e5e841ca812e0142a707a4f8', 'request_id': 'req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.811806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ec1530e5e841ca812e0142a707a4f8', 'request_id': 'req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.826757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] The token cache is being invalidate because project c5ec1530e5e841ca812e0142a707a4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:22.826757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.827371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.830124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1184/2377] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:22 2022] DELETE /identity/v3/projects/c5ec1530e5e841ca812e0142a707a4f8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:22.833970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:22.844612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf256d-33d4-48f5-a293-00644c4c487f service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:22.877845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:22.907357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:22.908016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:22.908285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:22.908564 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] PATH_INFO: `/v3/users/5b3ad557aa464597a4c5b3a8b2b48ac4` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:22.909735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3ad557aa464597a4c5b3a8b2b48ac4)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:22.911798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:22.964746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3ad557aa464597a4c5b3a8b2b48ac4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.977085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3ad557aa464597a4c5b3a8b2b48ac4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.977701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3ad557aa464597a4c5b3a8b2b48ac4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.985922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3ad557aa464597a4c5b3a8b2b48ac4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:22.999502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3ad557aa464597a4c5b3a8b2b48ac4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.005571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1194/2378] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:22 2022] DELETE /identity/v3/users/5b3ad557aa464597a4c5b3a8b2b48ac4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:46:23.300816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:23.303819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:23.320902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22beb8b-6da1-4eac-b4ec-9c4de1fb094b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:23.389931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:23.390617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:23.390900 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:23.391171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] PATH_INFO: `/v3/projects/322b626dd96d4b0fa8cb85b5693d85c3` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:23.392449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] RBAC: Authorizing `identity:delete_project(project_id=322b626dd96d4b0fa8cb85b5693d85c3)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:23.394404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:23.444300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:23.459602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322b626dd96d4b0fa8cb85b5693d85c3', 'request_id': 'req-c5e80794-1e0c-4ac8-8979-d9ea9034418a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.467318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322b626dd96d4b0fa8cb85b5693d85c3', 'request_id': 'req-c5e80794-1e0c-4ac8-8979-d9ea9034418a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.473805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322b626dd96d4b0fa8cb85b5693d85c3', 'request_id': 'req-c5e80794-1e0c-4ac8-8979-d9ea9034418a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.488514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] The token cache is being invalidate because project 322b626dd96d4b0fa8cb85b5693d85c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:23.488976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.490580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.492240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1185/2379] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:23 2022] DELETE /identity/v3/projects/322b626dd96d4b0fa8cb85b5693d85c3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:23.496249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:23.497806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:23.511212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba8720-f3a9-4552-90d6-51349aa1ff33 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:23.574652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:23.575197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:23.575656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:23.575939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] PATH_INFO: `/v3/users/7cfb83465c9e4459bf0eff30d6e008f7` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:23.577928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cfb83465c9e4459bf0eff30d6e008f7)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:23.580117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:23.648426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cfb83465c9e4459bf0eff30d6e008f7'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.653042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:23.660220 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cfb83465c9e4459bf0eff30d6e008f7'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.661033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cfb83465c9e4459bf0eff30d6e008f7'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.670266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cfb83465c9e4459bf0eff30d6e008f7'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.684936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ce057670-931a-4926-8a1d-47fc1e1cf9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cfb83465c9e4459bf0eff30d6e008f7'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:23.692007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1195/2380] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:23 2022] DELETE /identity/v3/users/7cfb83465c9e4459bf0eff30d6e008f7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:46:24.416389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:24.417933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:24.437124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e80794-1e0c-4ac8-8979-d9ea9034418a admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:24.504518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:24.505170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.505485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.505761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] PATH_INFO: `/v3/projects/5504a3307d41435aa42a7e0ac048993f` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.506931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5504a3307d41435aa42a7e0ac048993f)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:24.508860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:24.557627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5504a3307d41435aa42a7e0ac048993f', 'request_id': 'req-dfeddc48-c918-4ee6-9af5-f6303748b74d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:24.564175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5504a3307d41435aa42a7e0ac048993f', 'request_id': 'req-dfeddc48-c918-4ee6-9af5-f6303748b74d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:24.570587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5504a3307d41435aa42a7e0ac048993f', 'request_id': 'req-dfeddc48-c918-4ee6-9af5-f6303748b74d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:24.585701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] The token cache is being invalidate because project 5504a3307d41435aa42a7e0ac048993f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:24.585994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:24.586759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-dfeddc48-c918-4ee6-9af5-f6303748b74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:24.588273 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1186/2381] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:24 2022] DELETE /identity/v3/projects/5504a3307d41435aa42a7e0ac048993f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:24.595250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.595739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.595849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.615594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:24.616797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:24.680245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1196/2382] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:24.688797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.689088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.689368 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.709653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:24.710902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:24.719641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1187/2383] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:24.724084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:24.734289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73107a38-99a7-4c5f-b884-87282129fb70 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:24.739636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:24.740251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.740534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.740813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.741492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:46:24.741778 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:24.743452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:24.754927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1197/2384] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:46:24.759196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:24.768805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea92f3-aeab-475a-b020-bbc0100ad05d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:24.774068 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:24.774688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.774967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.775237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.775859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:24.777755 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:24.808418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1188/2385] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:24 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:24.812664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:24.822859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d489c01-9564-4939-8329-667dace1f8dc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:24.880580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:24.881225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.881510 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.881784 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.882454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:24.884411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:24.910044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1198/2386] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:24 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:24.914486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:24.925414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d391f-758b-4c7a-a6c6-6f82a2623083 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:24.932506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:24.933416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.933875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.934357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.935416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:24.938012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:24.949946 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1189/2387] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:24.954030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:24.965131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05082405-2845-4166-94bb-3b43e1ae963c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:24.971941 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e33a2-5c5d-421c-bde2-ff9db6fb5aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:24.972528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e33a2-5c5d-421c-bde2-ff9db6fb5aca admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:24.972804 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e33a2-5c5d-421c-bde2-ff9db6fb5aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:24.973069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e33a2-5c5d-421c-bde2-ff9db6fb5aca admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:24.973711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e33a2-5c5d-421c-bde2-ff9db6fb5aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:24.975360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e33a2-5c5d-421c-bde2-ff9db6fb5aca admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:24.990531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1199/2388] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:24.996851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.014050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226f783-96a2-4f67-b6e2-fe899e5882a2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.022428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.023091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.023374 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.023645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] PATH_INFO: `/v3/projects/ea55cd92ebfe46f4b3073f6eac70d7c6/users/cbea4793e92f4ec99d7d50ae2b2ebb83/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.037859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:25.043184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea55cd92ebfe46f4b3073f6eac70d7c6, user_id=cbea4793e92f4ec99d7d50ae2b2ebb83, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.045872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.064298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1190/2389] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:24 2022] PUT /identity/v3/projects/ea55cd92ebfe46f4b3073f6eac70d7c6/users/cbea4793e92f4ec99d7d50ae2b2ebb83/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:25.076692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.077062 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.077413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.108301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] MFA Rules not processed for user `cbea4793e92f4ec99d7d50ae2b2ebb83`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:25.109555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:25.136066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:25.180065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1200/2390] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:25.184770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.263457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4988f14-2a86-4cc5-85f0-93606f21981e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.268678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.269302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.269583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.269855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.270512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.272506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.300810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1191/2391] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:25 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:25.305903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.316923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4644da4-1d4b-4307-a423-017a45a08653 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.383862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.384486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.384772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.385071 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.385984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.387980 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.414173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1201/2392] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:25 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:25.418334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.429448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14dd9-69d0-41a7-94ac-f600d5842ba2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.435500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.436158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.436444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.436718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.437400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.439054 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.450524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1192/2393] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:25.454861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.465145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3d836-2123-4fe7-8462-c4d6562a7ff8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.470015 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.470717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.471016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.471278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] PATH_INFO: `/v3/projects/e86284d494c34913aaa6762fa0fe9d9a/users/4f0f340c8963420d840af10f25e8c332/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.486557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86284d494c34913aaa6762fa0fe9d9a, user_id=4f0f340c8963420d840af10f25e8c332, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.489211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.502709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1202/2394] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:25 2022] PUT /identity/v3/projects/e86284d494c34913aaa6762fa0fe9d9a/users/4f0f340c8963420d840af10f25e8c332/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:25.506936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.517315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e2ab-bc86-4800-b8ed-5ded65f0ea1c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.583370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a248da6-301f-4c8d-bac4-d84706cf33a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.583886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a248da6-301f-4c8d-bac4-d84706cf33a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.584176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a248da6-301f-4c8d-bac4-d84706cf33a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.584435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a248da6-301f-4c8d-bac4-d84706cf33a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.585133 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a248da6-301f-4c8d-bac4-d84706cf33a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.586786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a248da6-301f-4c8d-bac4-d84706cf33a4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.597156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1193/2395] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:46:25.602130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.613868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6ccc6-9845-4a82-a7a7-8b0ffc1c132f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.620599 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.621265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.621544 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.621821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] PATH_INFO: `/v3/projects/e86284d494c34913aaa6762fa0fe9d9a/users/4f0f340c8963420d840af10f25e8c332/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.624086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86284d494c34913aaa6762fa0fe9d9a, user_id=4f0f340c8963420d840af10f25e8c332, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.626786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.638568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1203/2396] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:25 2022] PUT /identity/v3/projects/e86284d494c34913aaa6762fa0fe9d9a/users/4f0f340c8963420d840af10f25e8c332/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:25.657268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.657565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.657839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.687543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] MFA Rules not processed for user `4f0f340c8963420d840af10f25e8c332`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:25.688674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:25.742823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:25.766010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1194/2397] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:25.777336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:25.779324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:25.845402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0628d-67bd-4845-ab4b-e28e83d4e89a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:25.849397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e711b-9f50-4808-b0ab-f19aa42e3cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:25.850004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e711b-9f50-4808-b0ab-f19aa42e3cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:25.850306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e711b-9f50-4808-b0ab-f19aa42e3cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:25.850608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e711b-9f50-4808-b0ab-f19aa42e3cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:25.880809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:25.908282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e711b-9f50-4808-b0ab-f19aa42e3cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:25.909736 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e711b-9f50-4808-b0ab-f19aa42e3cbe service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:25.919222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1204/2398] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:30.158944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:30.160467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:30.173061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23df1-0f7b-4e1f-bc88-3d781dd0a72b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:30.176657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f7146-076c-4ac3-83bf-987e74ddcbbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:30.177289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f7146-076c-4ac3-83bf-987e74ddcbbe service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.177568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f7146-076c-4ac3-83bf-987e74ddcbbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.177839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f7146-076c-4ac3-83bf-987e74ddcbbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.187945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f7146-076c-4ac3-83bf-987e74ddcbbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:30.189422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f7146-076c-4ac3-83bf-987e74ddcbbe service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:30.202773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1195/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:30.538403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1313e-b7d5-4bc0-ab58-93f004f3ee64 None None] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.538403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1313e-b7d5-4bc0-ab58-93f004f3ee64 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.539003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1313e-b7d5-4bc0-ab58-93f004f3ee64 None None] PATH_INFO: `/` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.542008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1205/2400] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:46:30.548995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a2850-b686-4d2d-afd1-ce3ba9c48c0d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.549386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a2850-b686-4d2d-afd1-ce3ba9c48c0d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.549761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a2850-b686-4d2d-afd1-ce3ba9c48c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.580786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d32a2850-b686-4d2d-afd1-ce3ba9c48c0d None None] MFA Rules not processed for user `a425b3b4b3f34217ba148c2f855e0068`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:30.582484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d32a2850-b686-4d2d-afd1-ce3ba9c48c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:30.645727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1196/2401] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:46:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:30.659388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1313e-b7d5-4bc0-ab58-93f004f3ee64 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:30.661125 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-60b1313e-b7d5-4bc0-ab58-93f004f3ee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:30.674399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1313e-b7d5-4bc0-ab58-93f004f3ee64 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:30.678003 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87bfc1-d7d7-4de0-a380-a07569c68318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:30.678641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87bfc1-d7d7-4de0-a380-a07569c68318 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.678917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87bfc1-d7d7-4de0-a380-a07569c68318 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.679186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87bfc1-d7d7-4de0-a380-a07569c68318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.686931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87bfc1-d7d7-4de0-a380-a07569c68318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:30.690265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87bfc1-d7d7-4de0-a380-a07569c68318 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:30.696399 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1206/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:30.781886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909994-2889-4c7c-9fdc-57e70b85f6dd None None] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.782211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909994-2889-4c7c-9fdc-57e70b85f6dd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.782496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909994-2889-4c7c-9fdc-57e70b85f6dd None None] PATH_INFO: `/` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.784003 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1197/2403] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 9 02:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 02:46:30.790676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.790965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.791240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.812109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] MFA Rules not processed for user `ad4a7ffff8f842969d782603cbb8fb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:30.813309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:30.885351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1207/2404] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 9 02:46:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:30.896402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909994-2889-4c7c-9fdc-57e70b85f6dd None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:30.898432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-db909994-2889-4c7c-9fdc-57e70b85f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:30.913265 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909994-2889-4c7c-9fdc-57e70b85f6dd None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:30.917423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b049d9e-ac19-4a3f-8996-5b0d28bbf945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:30.918023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b049d9e-ac19-4a3f-8996-5b0d28bbf945 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:30.918345 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b049d9e-ac19-4a3f-8996-5b0d28bbf945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:30.918617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b049d9e-ac19-4a3f-8996-5b0d28bbf945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:30.929094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b049d9e-ac19-4a3f-8996-5b0d28bbf945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:30.930634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b049d9e-ac19-4a3f-8996-5b0d28bbf945 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:30.941217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1198/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:35.179469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:35.255308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1da91-63e6-4873-be75-d2c605ce44b4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:35.260473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:35.261126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:35.261414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:35.261703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] PATH_INFO: `/v3/users/cbea4793e92f4ec99d7d50ae2b2ebb83` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:35.263069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbea4793e92f4ec99d7d50ae2b2ebb83)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:35.265124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:35.314037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbea4793e92f4ec99d7d50ae2b2ebb83'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.317622 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:35.322730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbea4793e92f4ec99d7d50ae2b2ebb83'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.323441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbea4793e92f4ec99d7d50ae2b2ebb83'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.332424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbea4793e92f4ec99d7d50ae2b2ebb83'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.349217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbea4793e92f4ec99d7d50ae2b2ebb83'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.355173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1208/2406] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:35 2022] DELETE /identity/v3/users/cbea4793e92f4ec99d7d50ae2b2ebb83 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:35.359666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:35.360174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:35.360274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:35.379989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:35.381189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:35.442685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1199/2407] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:35.452803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:35.454546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:35.515996 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905842a-71d1-4600-8698-aad76b2ea7f8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:35.519890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641224-f961-46be-8353-68003100386c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:35.520540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641224-f961-46be-8353-68003100386c service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:35.520824 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641224-f961-46be-8353-68003100386c service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:35.521149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641224-f961-46be-8353-68003100386c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:35.530742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3641224-f961-46be-8353-68003100386c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:35.532255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3641224-f961-46be-8353-68003100386c service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:35.542173 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1209/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:35.820895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:35.822202 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:35.838030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a884d-97f9-438f-bb6b-41eccf066245 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:35.844906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:35.845675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:35.845970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:35.846281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] PATH_INFO: `/v3/projects/ea55cd92ebfe46f4b3073f6eac70d7c6` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:35.847454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea55cd92ebfe46f4b3073f6eac70d7c6)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:35.849427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:35.898811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea55cd92ebfe46f4b3073f6eac70d7c6', 'request_id': 'req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.905077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea55cd92ebfe46f4b3073f6eac70d7c6', 'request_id': 'req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.911044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea55cd92ebfe46f4b3073f6eac70d7c6', 'request_id': 'req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.912739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:35.926399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] The token cache is being invalidate because project ea55cd92ebfe46f4b3073f6eac70d7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:35.926688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.927357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:35.928896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1200/2409] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:35 2022] DELETE /identity/v3/projects/ea55cd92ebfe46f4b3073f6eac70d7c6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:35.934657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641224-f961-46be-8353-68003100386c service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:35.944413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641224-f961-46be-8353-68003100386c service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.001607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.002350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.002632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.002903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] PATH_INFO: `/v3/users/4f0f340c8963420d840af10f25e8c332` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.004080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0f340c8963420d840af10f25e8c332)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.006096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.054811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0f340c8963420d840af10f25e8c332'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.063540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0f340c8963420d840af10f25e8c332'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.064111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0f340c8963420d840af10f25e8c332'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.072838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0f340c8963420d840af10f25e8c332'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.082248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:36.088841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-cc82a81d-d6af-4ba5-8351-59f655e5c48e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0f340c8963420d840af10f25e8c332'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.098375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1210/2410] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:35 2022] DELETE /identity/v3/users/4f0f340c8963420d840af10f25e8c332 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:46:36.379106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.380767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:36.399182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a973710-33d0-4a75-b9a3-8dd2f1a2a597 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.462285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.462922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.463194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.463463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] PATH_INFO: `/v3/projects/e86284d494c34913aaa6762fa0fe9d9a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.464579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86284d494c34913aaa6762fa0fe9d9a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.466528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.519042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86284d494c34913aaa6762fa0fe9d9a', 'request_id': 'req-4865d580-ddba-4538-ba88-6b489838a39c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.524743 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86284d494c34913aaa6762fa0fe9d9a', 'request_id': 'req-4865d580-ddba-4538-ba88-6b489838a39c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.530576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86284d494c34913aaa6762fa0fe9d9a', 'request_id': 'req-4865d580-ddba-4538-ba88-6b489838a39c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.544383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] The token cache is being invalidate because project e86284d494c34913aaa6762fa0fe9d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:36.544652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.545291 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4865d580-ddba-4538-ba88-6b489838a39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:36.546815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1201/2411] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:36 2022] DELETE /identity/v3/projects/e86284d494c34913aaa6762fa0fe9d9a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:36.552723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.553231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.553328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.575835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:36.577097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:36.636297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1211/2412] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:36.645167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.645458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.645735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.666997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:36.668225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:36.678177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1202/2413] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:46:36.682899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.696679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445cc6a-4692-4448-a907-e8296b71856a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.702326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.702933 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.703212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.703480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.704138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:46:36.704442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.706120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.718280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1212/2414] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:46:36.723021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.733414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7b6aa-8c1b-48ea-ab8c-47892ab3558a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.740324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.741010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.741263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.741564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.742193 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.744112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.776326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1203/2415] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:36 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:36.777621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.787850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44add59f-3fea-4e82-bf27-b3f60e393f65 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.846673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.847322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.847571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.847853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.848478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.850455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.874188 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1213/2416] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:36 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:36.878485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.889243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3443803-955b-4d8c-af3f-a643b018a022 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.895353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.895993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.896256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.896528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.897182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.898943 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.909566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1204/2417] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:36.913614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.924409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7255ab-86f3-4446-b9f4-eecdfbac6b39 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.929783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e092f-8c00-4739-9135-5f37eb6380ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.930444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e092f-8c00-4739-9135-5f37eb6380ff admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.930718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e092f-8c00-4739-9135-5f37eb6380ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.930984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e092f-8c00-4739-9135-5f37eb6380ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.931637 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e092f-8c00-4739-9135-5f37eb6380ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.933243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e092f-8c00-4739-9135-5f37eb6380ff admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.944760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1214/2418] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:36.948875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:36.959213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74689066-5204-476f-aa92-d6d8a424b32c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:36.964563 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:36.965222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:36.965499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:36.965766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] PATH_INFO: `/v3/projects/96e0d91af0474588853939bb04553a8f/users/9386fd90aed54586b84283f8e494b866/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:36.981167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e0d91af0474588853939bb04553a8f, user_id=9386fd90aed54586b84283f8e494b866, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:36.983881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:36.997781 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1205/2419] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:36 2022] PUT /identity/v3/projects/96e0d91af0474588853939bb04553a8f/users/9386fd90aed54586b84283f8e494b866/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:37.009411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.009693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.009993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.037356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] MFA Rules not processed for user `9386fd90aed54586b84283f8e494b866`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:37.038679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:37.085415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:37.107372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1215/2420] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:37.117819 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.138429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:37.186489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3411e-53d1-4ce2-b0aa-3a3241534ca0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.198431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.198431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.198431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.198431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.198844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.198844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:37.224178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1206/2421] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:37 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:37.228409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.242812 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa9a78-ab08-48c1-a8b6-f65069258cc3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.304745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.305448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.305772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.306144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.306872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.308866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:37.331779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1216/2422] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:37 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:37.335721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.346645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f243bc1-6d6e-420f-a00a-a0cbc647fcb7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.351311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.351924 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.352203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.352475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.353125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.354746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:37.366010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1207/2423] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:37.370250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.380836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9d135-f287-4453-8534-25d071cda665 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.386671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.387317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.387592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.387864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] PATH_INFO: `/v3/projects/74e1748ff44245f295ea355b5e0a6220/users/b4611de4007d44ad93b813e35d1bf8fb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.405229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e1748ff44245f295ea355b5e0a6220, user_id=b4611de4007d44ad93b813e35d1bf8fb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.409021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:37.426776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1217/2424] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:37 2022] PUT /identity/v3/projects/74e1748ff44245f295ea355b5e0a6220/users/b4611de4007d44ad93b813e35d1bf8fb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:46:37.430961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.442001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8eb7a-8721-4acd-95ba-34f23e83f5ec admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.521523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38714ea0-870f-47fa-af1f-1860d3add5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.523562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38714ea0-870f-47fa-af1f-1860d3add5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.523562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38714ea0-870f-47fa-af1f-1860d3add5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.523562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38714ea0-870f-47fa-af1f-1860d3add5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.523562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38714ea0-870f-47fa-af1f-1860d3add5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.525544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38714ea0-870f-47fa-af1f-1860d3add5f7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:37.537689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1208/2425] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:37.543043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.556779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e31f1-0b1e-405f-90f2-f309b3d58320 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.563504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.564807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.565871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.566481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] PATH_INFO: `/v3/projects/74e1748ff44245f295ea355b5e0a6220/users/b4611de4007d44ad93b813e35d1bf8fb/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.569982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e1748ff44245f295ea355b5e0a6220, user_id=b4611de4007d44ad93b813e35d1bf8fb, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.573810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:37.591867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1218/2426] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:37 2022] PUT /identity/v3/projects/74e1748ff44245f295ea355b5e0a6220/users/b4611de4007d44ad93b813e35d1bf8fb/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:37.675321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.675707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.675976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.723319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] MFA Rules not processed for user `b4611de4007d44ad93b813e35d1bf8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:37.727448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:37.833410 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1209/2427] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:37.847406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:37.849362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:37.923006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d301619-c323-4959-a6b5-2adaf35f19dc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:37.926651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7e879-a096-4e9a-aa34-1d9e615568dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:37.927267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7e879-a096-4e9a-aa34-1d9e615568dc service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:37.927557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7e879-a096-4e9a-aa34-1d9e615568dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:37.927831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7e879-a096-4e9a-aa34-1d9e615568dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:37.985766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:37.991239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7e879-a096-4e9a-aa34-1d9e615568dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:37.993246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7e879-a096-4e9a-aa34-1d9e615568dc service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:38.002302 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1219/2428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:46:40.628885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:40.643634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:40.694997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d995-d363-4f1f-aa51-cdd273e76807 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:40.700105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:40.700732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:40.701010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:40.701281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] PATH_INFO: `/v3/users/9386fd90aed54586b84283f8e494b866` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:40.702488 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9386fd90aed54586b84283f8e494b866)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:40.704411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:40.756984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9386fd90aed54586b84283f8e494b866'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:40.757674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9386fd90aed54586b84283f8e494b866'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:40.765449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9386fd90aed54586b84283f8e494b866'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:40.772443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9386fd90aed54586b84283f8e494b866'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:40.784502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9386fd90aed54586b84283f8e494b866'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:40.791189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1210/2429] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:40 2022] DELETE /identity/v3/users/9386fd90aed54586b84283f8e494b866 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:40.795988 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:40.796446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:40.796446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:40.816706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:40.817931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:40.892095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1220/2430] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:40.899303 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:40.900904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:40.964386 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274741-aec6-4dfa-801b-2e2847749987 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:40.968223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:40.968848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:40.969123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:40.969390 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:40.978958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:40.980472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:40.990903 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1211/2431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:41.284287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:41.285920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:41.300931 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89a6a-2ea0-45ba-b284-9f67f3e29f94 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:41.307919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:41.308677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:41.308837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:41.309112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] PATH_INFO: `/v3/projects/96e0d91af0474588853939bb04553a8f` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:41.310734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e0d91af0474588853939bb04553a8f)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:41.312409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:41.372902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e0d91af0474588853939bb04553a8f', 'request_id': 'req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.381330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e0d91af0474588853939bb04553a8f', 'request_id': 'req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.388135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e0d91af0474588853939bb04553a8f', 'request_id': 'req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.398396 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] The token cache is being invalidate because project 96e0d91af0474588853939bb04553a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:41.398684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.399321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.400836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1221/2432] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:41 2022] DELETE /identity/v3/projects/96e0d91af0474588853939bb04553a8f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:41.405339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:41.416540 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b34bf8-dc2b-4cad-8b4a-90467d5c7cc1 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:41.451641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:41.477190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:41.477756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:41.478041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:41.478347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] PATH_INFO: `/v3/users/b4611de4007d44ad93b813e35d1bf8fb` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:41.479478 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4611de4007d44ad93b813e35d1bf8fb)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:41.481447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:41.529614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4611de4007d44ad93b813e35d1bf8fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.530325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4611de4007d44ad93b813e35d1bf8fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.537859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4611de4007d44ad93b813e35d1bf8fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.544674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4611de4007d44ad93b813e35d1bf8fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.556896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-cd49d550-8add-42aa-bfab-8c59087c8bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4611de4007d44ad93b813e35d1bf8fb'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.563081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1212/2433] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:46:41 2022] DELETE /identity/v3/users/b4611de4007d44ad93b813e35d1bf8fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:41.846668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:41.848301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:41.865040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31e3b3-cb22-4bd6-839c-5b766bf224ee admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:41.927150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:41.927820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:41.928094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:41.928362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] PATH_INFO: `/v3/projects/74e1748ff44245f295ea355b5e0a6220` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:41.929581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e1748ff44245f295ea355b5e0a6220)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:41.931538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:41.952030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:41.980347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e1748ff44245f295ea355b5e0a6220', 'request_id': 'req-adc1c027-d7a8-4c92-a531-b36e1ff126f1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.988434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e1748ff44245f295ea355b5e0a6220', 'request_id': 'req-adc1c027-d7a8-4c92-a531-b36e1ff126f1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:41.994616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e1748ff44245f295ea355b5e0a6220', 'request_id': 'req-adc1c027-d7a8-4c92-a531-b36e1ff126f1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:42.005178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] The token cache is being invalidate because project 74e1748ff44245f295ea355b5e0a6220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:46:42.005456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:42.006115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-adc1c027-d7a8-4c92-a531-b36e1ff126f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:46:42.007674 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1222/2434] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:46:41 2022] DELETE /identity/v3/projects/74e1748ff44245f295ea355b5e0a6220 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:42.013022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.013513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.013513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.034134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:42.035430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:42.096299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1213/2435] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:42.104893 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.105250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.105602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.128806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:42.130056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:42.138423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1223/2436] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:42.142810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.153118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133e484-7485-419e-8c60-515466a87ad8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.159105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.159719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.160001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.160275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.160930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:46:42.161232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.162880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.175413 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1214/2437] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:46:42.180013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.189811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35398a-fa08-47b7-9219-a32d4d6b1db9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.196137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.196740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.197056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.197293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.197928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.199887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.229781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1224/2438] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:46:42 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:42.234058 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.244672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c86d-1321-49ec-a5b1-ea7ae24bad92 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.284920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:42.303039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.303712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.303993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.304258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.305018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.306988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.328848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1215/2439] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:46:42 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:46:42.333115 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.343934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8feafd-07f1-4bb0-be3d-93daca7bf3c0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.349427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.350014 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.350363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.350633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.351294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.353049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.363698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1225/2440] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:42.368037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.378823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13b6bb-9443-4fce-8ee5-f67c355f047e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.384169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fd667-4b22-499c-a8ff-3012397dd790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.384777 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fd667-4b22-499c-a8ff-3012397dd790 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.385060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fd667-4b22-499c-a8ff-3012397dd790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.385342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fd667-4b22-499c-a8ff-3012397dd790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.385991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fd667-4b22-499c-a8ff-3012397dd790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.387619 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fd667-4b22-499c-a8ff-3012397dd790 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.399314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1216/2441] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:46:42.403316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.414138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25591-0e03-4115-8941-d99f4c433355 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.423725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.424402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.424677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.424975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] PATH_INFO: `/v3/projects/a4f0136e9c644c85959d325cb47c72fa/users/ef5177aad29842a6a73b747f957e3f78/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.440536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f0136e9c644c85959d325cb47c72fa, user_id=ef5177aad29842a6a73b747f957e3f78, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.443366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.461479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1226/2442] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:46:42 2022] PUT /identity/v3/projects/a4f0136e9c644c85959d325cb47c72fa/users/ef5177aad29842a6a73b747f957e3f78/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:46:42.471983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.472349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.472692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.501979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] MFA Rules not processed for user `ef5177aad29842a6a73b747f957e3f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:46:42.503266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:42.570376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1217/2443] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:46:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:46:42.583028 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:46:42.584953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:46:42.650140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5900-dd41-4dd1-a81f-20b774bf50c2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:46:42.653699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba040a2b-2217-4557-84a3-20d5c13d17c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:46:42.654345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040a2b-2217-4557-84a3-20d5c13d17c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:46:42.654621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040a2b-2217-4557-84a3-20d5c13d17c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:46:42.654890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040a2b-2217-4557-84a3-20d5c13d17c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:46:42.662972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba040a2b-2217-4557-84a3-20d5c13d17c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:46:42.664417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba040a2b-2217-4557-84a3-20d5c13d17c4 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:46:42.672967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1227/2444] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:46:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:46:52.566848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:46:52.669463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:02.641898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:02.702496 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35df0-8a4c-4b2e-acf7-6d7a1cffdad3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:02.706925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:02.707555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:02.707831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:02.708101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] PATH_INFO: `/v3/users/ef5177aad29842a6a73b747f957e3f78` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:02.709232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5177aad29842a6a73b747f957e3f78)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:02.711210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:02.760224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5177aad29842a6a73b747f957e3f78'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:02.760875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5177aad29842a6a73b747f957e3f78'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:02.768922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5177aad29842a6a73b747f957e3f78'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:02.775805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5177aad29842a6a73b747f957e3f78'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:02.788769 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5177aad29842a6a73b747f957e3f78'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:02.795397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1218/2445] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:02 2022] DELETE /identity/v3/users/ef5177aad29842a6a73b747f957e3f78 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:02.800049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:02.800594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:02.800594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:02.821090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:02.822439 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:02.890239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1228/2446] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:02.897156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:02.898786 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:02.955197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778fc5-b77c-49a5-a7b6-c8dc606f2c25 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:02.958936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:02.959554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:02.959838 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:02.960114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:02.970246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:02.971729 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:02.981968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1219/2447] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:03.297375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:03.299616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:03.332942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bdba4-640f-49ce-9bad-a9e1c316ad18 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:03.341412 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:03.342075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:03.342385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:03.342656 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] PATH_INFO: `/v3/projects/a4f0136e9c644c85959d325cb47c72fa` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:03.343836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f0136e9c644c85959d325cb47c72fa)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:03.345788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:03.404630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:03.410396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:03.419879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f0136e9c644c85959d325cb47c72fa', 'request_id': 'req-67585dcf-1af6-4f0f-9d35-2ce766d7d071'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:03.422356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5435b11-24bd-4c31-ad3b-bc6a8d4a4cdf service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:03.428795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f0136e9c644c85959d325cb47c72fa', 'request_id': 'req-67585dcf-1af6-4f0f-9d35-2ce766d7d071'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:03.435709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f0136e9c644c85959d325cb47c72fa', 'request_id': 'req-67585dcf-1af6-4f0f-9d35-2ce766d7d071'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:03.443545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:03.446022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] The token cache is being invalidate because project a4f0136e9c644c85959d325cb47c72fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:03.446350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:03.446982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:03.448476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1229/2448] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:03 2022] DELETE /identity/v3/projects/a4f0136e9c644c85959d325cb47c72fa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:03.480754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:03.481762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:03.482023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:03.482330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:03.484421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:03.550724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:03.552197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:03.562932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1220/2449] 10.222.0.20 () {60 vars in 1396 bytes} [Tue Aug 9 02:47:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:05.838371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:05.840555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:05.856338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585dcf-1af6-4f0f-9d35-2ce766d7d071 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:05.860158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:05.860811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:05.861068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:05.861343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:05.863702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:05.931474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:05.932971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:05.942835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1230/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:06.117510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:06.119095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:06.129912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358937-ef6c-4d43-a802-ebfa06abdd78 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:06.133765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:06.134384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:06.134859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:06.135128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:06.136919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:06.148945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:06.150395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:06.160520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1221/2451] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:09.581498 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:09.583853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:09.613160 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:09.648936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:09.651087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:09.666266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e08de3-5386-47a2-a881-6349bca53f2b service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:09.675963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:09.676742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:09.677091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:09.677429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] PATH_INFO: `/v3/users/43232b9b191047f9a3e6ebab7ad36f02` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:09.682026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43232b9b191047f9a3e6ebab7ad36f02)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:09.686537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:09.697628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb9cc2-0859-4609-b30e-765bd968abbd service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:09.703611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:09.704235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:09.704520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:09.704793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:09.706961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:09.721583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:09.723743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:09.743895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1231/2452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:09.743895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:09.747242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:09.751793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43232b9b191047f9a3e6ebab7ad36f02'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.752869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43232b9b191047f9a3e6ebab7ad36f02'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.764253 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f086c7-d753-46b8-a7f4-a7497dc99100 service placement] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:09.767339 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43232b9b191047f9a3e6ebab7ad36f02'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.775596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:09.776821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:09.777448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:09.781740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] PATH_INFO: `/v3/users/2aaa8a7f386e4ccc9ab5c356cdf1704f` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:09.783873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aaa8a7f386e4ccc9ab5c356cdf1704f)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:09.786793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:09.800146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43232b9b191047f9a3e6ebab7ad36f02'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.826598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43232b9b191047f9a3e6ebab7ad36f02'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.835100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1222/2453] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:09 2022] DELETE /identity/v3/users/43232b9b191047f9a3e6ebab7ad36f02 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:09.900053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aaa8a7f386e4ccc9ab5c356cdf1704f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.918725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aaa8a7f386e4ccc9ab5c356cdf1704f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.921842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aaa8a7f386e4ccc9ab5c356cdf1704f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.935304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aaa8a7f386e4ccc9ab5c356cdf1704f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.950590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aaa8a7f386e4ccc9ab5c356cdf1704f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:09.959214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1232/2454] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:09 2022] DELETE /identity/v3/users/2aaa8a7f386e4ccc9ab5c356cdf1704f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:10.044322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.046020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.102805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:10.129834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bd828-edb6-4435-94c7-b5ce6c4b9cf3 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.170781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.172680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.204661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.205406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.205752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.206093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] PATH_INFO: `/v3/projects/8ce26c854fc24cadb2be5ebd441b8f3a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.207301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce26c854fc24cadb2be5ebd441b8f3a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.209366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.241940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dc318-f93d-41ad-ae13-55fc30a798de admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.246696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.247333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.247609 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.247876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] PATH_INFO: `/v3/projects/467b42c2bec74fcbbf90d007a6e00096` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.248983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=467b42c2bec74fcbbf90d007a6e00096)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.251051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.258787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce26c854fc24cadb2be5ebd441b8f3a', 'request_id': 'req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.274095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce26c854fc24cadb2be5ebd441b8f3a', 'request_id': 'req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.275221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:10.280753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce26c854fc24cadb2be5ebd441b8f3a', 'request_id': 'req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.295397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] The token cache is being invalidate because project 8ce26c854fc24cadb2be5ebd441b8f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:10.295689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.296380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5a4ce1aa-7f6b-4c09-ac7f-139a9e400b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.298290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1223/2455] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:10 2022] DELETE /identity/v3/projects/8ce26c854fc24cadb2be5ebd441b8f3a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:10.302234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467b42c2bec74fcbbf90d007a6e00096', 'request_id': 'req-762a3e53-36c6-4301-bd17-6dd24bcfb899'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.303184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb3ef2-218c-4ff4-997d-26c35d4ddda5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.303481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb3ef2-218c-4ff4-997d-26c35d4ddda5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.303802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb3ef2-218c-4ff4-997d-26c35d4ddda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.311733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467b42c2bec74fcbbf90d007a6e00096', 'request_id': 'req-762a3e53-36c6-4301-bd17-6dd24bcfb899'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.318356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467b42c2bec74fcbbf90d007a6e00096', 'request_id': 'req-762a3e53-36c6-4301-bd17-6dd24bcfb899'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.324085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-fddb3ef2-218c-4ff4-997d-26c35d4ddda5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:10.325322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fddb3ef2-218c-4ff4-997d-26c35d4ddda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.329355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] The token cache is being invalidate because project 467b42c2bec74fcbbf90d007a6e00096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:10.329768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.330488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-762a3e53-36c6-4301-bd17-6dd24bcfb899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:10.332039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1233/2456] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:10 2022] DELETE /identity/v3/projects/467b42c2bec74fcbbf90d007a6e00096 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:10.336811 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e9f87-d714-489f-8a12-373e3f9b7666 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.337196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e9f87-d714-489f-8a12-373e3f9b7666 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.337567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e9f87-d714-489f-8a12-373e3f9b7666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.357087 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-a76e9f87-d714-489f-8a12-373e3f9b7666 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:10.358407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a76e9f87-d714-489f-8a12-373e3f9b7666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.385529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1224/2457] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:10.394170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.394484 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.394766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.414642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:10.416073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.418775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1234/2458] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:10.426813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.427217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.427595 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.431330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1225/2459] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:10.432421 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.444408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d47512-4ec2-4b65-a314-4f4287120528 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.448927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:10.450146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.450866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.450975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.451276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.451614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.452346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:10.452721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.454408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.456351 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:10.463009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1235/2460] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:10.467268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.473109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1226/2461] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:10.477649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.478140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6e3ba-51cf-4474-a8a5-7cb4c1577c28 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.483716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.484594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.484851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.485110 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.486427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:10.486740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.489712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.490301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ceaa30-d30c-4b64-a73f-d1f68982a9c8 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.494999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.495594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.495881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.496152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.496808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.498902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.506923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1236/2462] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:10.508307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.521162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb155-c86e-484f-a297-0ffcf174afe3 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.527096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.527727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.528001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.528314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.528909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.530908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.533301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1227/2463] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:10.537318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.564541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1237/2464] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:47:10.565597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.608636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1ef78-2cc6-4c0e-bd61-24aafffc69a1 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.635838 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d45fd-590e-4c57-b14c-a8eea7952cc6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.640858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.641473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.642134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.642134 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.642693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.644617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.661874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:10.669160 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.669857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.670235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.670413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1238/2465] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:10.670523 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.671295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.672689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.673628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.683547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9c3e6-50d0-4413-8a88-2dda1b529a35 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.689261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.690131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.690468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.690851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.691815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.693696 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.702031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1228/2466] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:10.703175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.707418 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1239/2467] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:10.708373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.714989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817862-10f5-4038-8757-8562a0f2b16c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.719407 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e0b60-86fe-4247-b377-e9491d86b4eb admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.719657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.720313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.720542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.720811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.721459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.723085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.724178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.724773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.725056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.725338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.725978 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.727592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.736369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1229/2468] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:10.737747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.741694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1240/2469] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:10.742665 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.747448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b8798-5fdb-446e-9000-8519da96eb75 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.752023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.752172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb9c1f-b632-4e7e-8f18-80f58e4f07f6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.752609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.752892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.753170 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.753804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.755416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.757081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adc6e7-a77a-4cda-b496-750bf4104110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.758041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adc6e7-a77a-4cda-b496-750bf4104110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.758365 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adc6e7-a77a-4cda-b496-750bf4104110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.758640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adc6e7-a77a-4cda-b496-750bf4104110 admin admin] PATH_INFO: `/v3/projects/3dd9b0c6a3054114b6f7b5db74101f15/users/57b1873a17c0452b96043e364effb50c/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.765993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1230/2470] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:10.770072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.773956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adc6e7-a77a-4cda-b496-750bf4104110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd9b0c6a3054114b6f7b5db74101f15, user_id=57b1873a17c0452b96043e364effb50c, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.776700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adc6e7-a77a-4cda-b496-750bf4104110 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.780039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebf58-277b-4ba0-bb4b-94b4068c42f4 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:10.785078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf40c4e-5207-4923-b4be-2ff31aec971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:10.785788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf40c4e-5207-4923-b4be-2ff31aec971a admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.786028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf40c4e-5207-4923-b4be-2ff31aec971a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.786314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf40c4e-5207-4923-b4be-2ff31aec971a admin admin] PATH_INFO: `/v3/projects/1ec3db1789a249459770141ece7f9e60/users/4fe680af4dff41d8ae2680cc84aa01d1/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.792286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1241/2471] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:10 2022] PUT /identity/v3/projects/3dd9b0c6a3054114b6f7b5db74101f15/users/57b1873a17c0452b96043e364effb50c/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:10.802579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf40c4e-5207-4923-b4be-2ff31aec971a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec3db1789a249459770141ece7f9e60, user_id=4fe680af4dff41d8ae2680cc84aa01d1, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:10.804534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.804897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.805211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf40c4e-5207-4923-b4be-2ff31aec971a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:10.805306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.826990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1231/2472] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:10 2022] PUT /identity/v3/projects/1ec3db1789a249459770141ece7f9e60/users/4fe680af4dff41d8ae2680cc84aa01d1/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:10.838201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:10.838439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:10.838720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:10.839569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] MFA Rules not processed for user `57b1873a17c0452b96043e364effb50c`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:10.840789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.867973 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] MFA Rules not processed for user `4fe680af4dff41d8ae2680cc84aa01d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:10.869195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.878541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:10.910423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1242/2473] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:10.921592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.922886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.937626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1232/2474] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:10.949013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:10.950450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:10.998219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db210b7-e87b-4b33-9711-cf0239445123 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:11.004212 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:11.004831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:11.005108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:11.005382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:11.013020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:11.014745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:11.023492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699e0f-5aa6-4991-be07-1f163aff258a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:11.023897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1243/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:47:11.029275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36face70-612f-4b7f-a947-e27c5b87adad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:11.030242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36face70-612f-4b7f-a947-e27c5b87adad service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:11.030242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36face70-612f-4b7f-a947-e27c5b87adad service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:11.030542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36face70-612f-4b7f-a947-e27c5b87adad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:11.039597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36face70-612f-4b7f-a947-e27c5b87adad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:11.041178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36face70-612f-4b7f-a947-e27c5b87adad service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:11.050408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1233/2476] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:47:11.447288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:11.481320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:11.518386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1cac-f708-430b-b46d-e79d6a213848 service nova] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:11.523152 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:11.523796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:11.524079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:11.524393 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] PATH_INFO: `/v3/users/4fe680af4dff41d8ae2680cc84aa01d1` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:11.525565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe680af4dff41d8ae2680cc84aa01d1)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:11.527549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:11.585295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe680af4dff41d8ae2680cc84aa01d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:11.593926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe680af4dff41d8ae2680cc84aa01d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:11.594605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe680af4dff41d8ae2680cc84aa01d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:11.601444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe680af4dff41d8ae2680cc84aa01d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:11.614445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe680af4dff41d8ae2680cc84aa01d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:11.620375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1244/2477] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:11 2022] DELETE /identity/v3/users/4fe680af4dff41d8ae2680cc84aa01d1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:11.624952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:11.625235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:11.625509 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:11.645684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:11.646955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:11.706597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1234/2478] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:11.717763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:11.719513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:11.787223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5d556-70d0-4ed3-94a4-1f6bd0160920 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:11.791196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ce6fc-855b-40d3-80e6-b289382e5f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:11.791810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce6fc-855b-40d3-80e6-b289382e5f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:11.792149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce6fc-855b-40d3-80e6-b289382e5f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:11.792427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce6fc-855b-40d3-80e6-b289382e5f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:11.801722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ce6fc-855b-40d3-80e6-b289382e5f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:11.803323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ce6fc-855b-40d3-80e6-b289382e5f67 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:11.814704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1245/2479] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:12.096296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.097804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:12.115334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85347f97-6542-43de-8b5c-8423023717f9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.120697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.121326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.121565 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.121837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] PATH_INFO: `/v3/projects/1ec3db1789a249459770141ece7f9e60` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.123094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec3db1789a249459770141ece7f9e60)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.125084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.160917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:12.173371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec3db1789a249459770141ece7f9e60', 'request_id': 'req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:12.179544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec3db1789a249459770141ece7f9e60', 'request_id': 'req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:12.185105 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec3db1789a249459770141ece7f9e60', 'request_id': 'req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:12.198375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] The token cache is being invalidate because project 1ec3db1789a249459770141ece7f9e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:12.198708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:12.199403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a54ea46f-77c8-4e0a-9a8f-e3b8d5c03577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:12.200963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1235/2480] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:12 2022] DELETE /identity/v3/projects/1ec3db1789a249459770141ece7f9e60 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:12.206016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.206335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.206611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.226300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:12.227512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:12.273042 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:12.288959 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1246/2481] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:12.297013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.297305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.297578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.319239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:12.320504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:12.329037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1236/2482] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:12.333363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.343328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07255f-7262-46ab-86df-4bcc5e4abb5c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.348375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.348991 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.349271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.350083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.350268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:12.350577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.355783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.369336 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1247/2483] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:12.373957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.384185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e9eb-11ff-4dae-b136-7418fa302cce None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.388933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.389545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.389851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.390120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.390807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.392698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.424092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1237/2484] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:12.434389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.445502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb314c-c9be-42b7-876c-25a5460ae8f2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.509364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.509987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.510303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.510574 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.511196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.513310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.538234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1248/2485] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:12.541967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.552805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf6ca7-3b9d-4e38-98a8-dffaa6a18bf8 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.558226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.558875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.559188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.559460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.560117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.561918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.574203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1238/2486] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:12.576769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.587086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13de417-a9de-4aaf-b813-8e0c4e4d956f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.591908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27fc9d-ce8f-4f08-a9d0-e4b1eabd5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.592521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27fc9d-ce8f-4f08-a9d0-e4b1eabd5dff admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.592800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27fc9d-ce8f-4f08-a9d0-e4b1eabd5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.593132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27fc9d-ce8f-4f08-a9d0-e4b1eabd5dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.593842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27fc9d-ce8f-4f08-a9d0-e4b1eabd5dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.595474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27fc9d-ce8f-4f08-a9d0-e4b1eabd5dff admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.606586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1249/2487] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:12.610879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.622064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a6e38-f024-42e6-af08-de4a1fa628e4 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.627057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.627724 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.628013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.628274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] PATH_INFO: `/v3/projects/0ac60e337fda427790fdee195027f174/users/eb497c9ae90f4c248c6bc6fd38d4ce38/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.643331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac60e337fda427790fdee195027f174, user_id=eb497c9ae90f4c248c6bc6fd38d4ce38, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.646072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.658952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1239/2488] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:12 2022] PUT /identity/v3/projects/0ac60e337fda427790fdee195027f174/users/eb497c9ae90f4c248c6bc6fd38d4ce38/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:12.670819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.671121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.671397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.703817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] MFA Rules not processed for user `eb497c9ae90f4c248c6bc6fd38d4ce38`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:12.705177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:12.773945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1250/2489] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:12.778543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.839722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70426-2dde-4a19-85e5-6b1ad86f67d0 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.844859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.845399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.845663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.845930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.846655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.848554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.877143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1240/2490] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:12.881474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.893121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb2be8-b92f-41f3-96a8-a46442b479a9 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:12.936741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:12.955502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:12.956114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:12.956391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:12.956713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:12.957334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:12.959305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:12.982626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1251/2491] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:12 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:12.987028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:12.998594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5e676-d689-454a-bd97-1589368fef64 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:13.004402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:13.004996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.005283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.005549 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.006217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:13.007824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:13.018682 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1241/2492] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:13.023005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:13.033492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5ea4b-888c-470e-a330-ad4621202b84 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:13.041155 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:13.042093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.042445 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.042719 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] PATH_INFO: `/v3/projects/7a5cffa94c5b4c15a38ebcd2909b1679/users/36b57f0d07224be6800445e23d7ccaa4/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.060296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5cffa94c5b4c15a38ebcd2909b1679, user_id=36b57f0d07224be6800445e23d7ccaa4, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:13.063024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:13.077735 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1252/2493] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:13 2022] PUT /identity/v3/projects/7a5cffa94c5b4c15a38ebcd2909b1679/users/36b57f0d07224be6800445e23d7ccaa4/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:13.081766 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:13.091716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7329d-60b7-4ec4-b0d4-e130eb9b016c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:13.121683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:13.151192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075b322-e6e0-47d2-b037-63aaa26f8475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:13.151811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075b322-e6e0-47d2-b037-63aaa26f8475 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.152088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075b322-e6e0-47d2-b037-63aaa26f8475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.152357 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075b322-e6e0-47d2-b037-63aaa26f8475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.153005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075b322-e6e0-47d2-b037-63aaa26f8475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:13.154661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075b322-e6e0-47d2-b037-63aaa26f8475 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:13.164500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1242/2494] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:13.168863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:13.179541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493b27-e149-496d-9346-b23c101191b2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:13.184730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:13.185375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.185655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.185914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] PATH_INFO: `/v3/projects/7a5cffa94c5b4c15a38ebcd2909b1679/users/36b57f0d07224be6800445e23d7ccaa4/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.188099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5cffa94c5b4c15a38ebcd2909b1679, user_id=36b57f0d07224be6800445e23d7ccaa4, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:13.190760 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:13.201455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1253/2495] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:13 2022] PUT /identity/v3/projects/7a5cffa94c5b4c15a38ebcd2909b1679/users/36b57f0d07224be6800445e23d7ccaa4/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:13.211712 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.211994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.212262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.240391 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] MFA Rules not processed for user `36b57f0d07224be6800445e23d7ccaa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:13.241661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:13.310595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1243/2496] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:13.322817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:13.324686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:13.519331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1048-f598-403d-9f31-b99d7dd5ee7c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:13.523583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:13.524228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.524500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.524764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.534292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:13.535700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:13.545292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1254/2497] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:13.626473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:13.637129 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa94dbe-474e-43ef-b2e1-390b64e92863 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:13.641657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07709df-52b5-4be0-8daa-6b13459f6620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:13.642325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07709df-52b5-4be0-8daa-6b13459f6620 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:13.642611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07709df-52b5-4be0-8daa-6b13459f6620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:13.642886 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07709df-52b5-4be0-8daa-6b13459f6620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:13.702475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07709df-52b5-4be0-8daa-6b13459f6620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:13.703939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07709df-52b5-4be0-8daa-6b13459f6620 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:13.713114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1244/2498] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:18.392236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:18.392737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:18.458819 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6d92-530a-40ae-82e2-f28f22a6e42c service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:18.463239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:18.463866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:18.464171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:18.464410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] PATH_INFO: `/v3/users/57b1873a17c0452b96043e364effb50c` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:18.465523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b1873a17c0452b96043e364effb50c)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:18.467591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:18.512423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:18.518997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b1873a17c0452b96043e364effb50c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:18.526803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b1873a17c0452b96043e364effb50c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:18.527404 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b1873a17c0452b96043e364effb50c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:18.534195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b1873a17c0452b96043e364effb50c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:18.545967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b1873a17c0452b96043e364effb50c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:18.552186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1255/2499] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:18 2022] DELETE /identity/v3/users/57b1873a17c0452b96043e364effb50c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:18.556999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:18.557534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:18.557749 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:18.577120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:18.578348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:18.626022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:18.641207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1245/2500] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:18.651046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:18.652493 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:18.717240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41eceef-1c93-4e56-9a4d-e58f5d8473fe admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:18.721443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9286847-83c0-4ca2-9931-6a0372288495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:18.722045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9286847-83c0-4ca2-9931-6a0372288495 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:18.722357 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9286847-83c0-4ca2-9931-6a0372288495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:18.722629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9286847-83c0-4ca2-9931-6a0372288495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:18.731972 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9286847-83c0-4ca2-9931-6a0372288495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:18.733431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9286847-83c0-4ca2-9931-6a0372288495 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:18.743432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1256/2501] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:19.025626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.027444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:19.040447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c756be5-de5d-4540-aad4-ff5e034b7e88 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.045919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.046582 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.046862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.047134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] PATH_INFO: `/v3/projects/3dd9b0c6a3054114b6f7b5db74101f15` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.048491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd9b0c6a3054114b6f7b5db74101f15)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.050343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.102800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd9b0c6a3054114b6f7b5db74101f15', 'request_id': 'req-b6e746ec-d783-4600-b6f3-ab2691878da2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:19.109992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd9b0c6a3054114b6f7b5db74101f15', 'request_id': 'req-b6e746ec-d783-4600-b6f3-ab2691878da2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:19.118608 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd9b0c6a3054114b6f7b5db74101f15', 'request_id': 'req-b6e746ec-d783-4600-b6f3-ab2691878da2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:19.134929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] The token cache is being invalidate because project 3dd9b0c6a3054114b6f7b5db74101f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:19.135219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:19.135854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b6e746ec-d783-4600-b6f3-ab2691878da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:19.137501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1246/2502] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:19 2022] DELETE /identity/v3/projects/3dd9b0c6a3054114b6f7b5db74101f15 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:19.142731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.143022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.143329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.162994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:19.164202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:19.228117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1257/2503] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:19.236181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.236461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.236736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.257606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:19.258865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:19.267595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1247/2504] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:19.272053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.283659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841eda-d8e1-4b93-bc3c-78da99de331e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.289568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.290182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.290478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.290747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.291410 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:19.291711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.293300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.304936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1258/2505] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:47:19.309508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.319425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25999fd2-d946-4ae4-b115-1ba54aa644b5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.325148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.325778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.326060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.326370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.327002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.328926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.359600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1248/2506] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:19 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:19.366730 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.380724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c2005-2369-41f3-ac47-28478eba22a4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.432270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:19.448539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.449159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.449452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.449717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.450383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.452284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.474766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1259/2507] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:19 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:19.478992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.490528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68935957-3238-4903-8c67-d03957019f31 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.495183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.495800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.496083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.496355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.497035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.498694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.509428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1249/2508] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:19.513634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.526504 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a1766-8805-4017-845d-cbdf25fcdc4d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.531266 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef540a-5294-40c4-8722-0f86ed5eca51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.531876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef540a-5294-40c4-8722-0f86ed5eca51 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.532288 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef540a-5294-40c4-8722-0f86ed5eca51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.532610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef540a-5294-40c4-8722-0f86ed5eca51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.533346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef540a-5294-40c4-8722-0f86ed5eca51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.534993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef540a-5294-40c4-8722-0f86ed5eca51 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.546194 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1260/2509] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:19.549424 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.559630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b608af-ae8a-4ae7-a0f5-43ca5b5a7167 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.564237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.564891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.565182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.565456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] PATH_INFO: `/v3/projects/96979d19de8a4c479a6538a6b35f8798/users/e8ee2ac0e2de411abec414e3dee9bedb/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.580018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] RBAC: Authorizing `identity:create_grant(project_id=96979d19de8a4c479a6538a6b35f8798, user_id=e8ee2ac0e2de411abec414e3dee9bedb, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.582716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.597164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1250/2510] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:19 2022] PUT /identity/v3/projects/96979d19de8a4c479a6538a6b35f8798/users/e8ee2ac0e2de411abec414e3dee9bedb/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:19.607703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b799c6-aac2-4fe3-9f48-5b6282bcd415 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.608007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b799c6-aac2-4fe3-9f48-5b6282bcd415 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.608272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b799c6-aac2-4fe3-9f48-5b6282bcd415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.636968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-63b799c6-aac2-4fe3-9f48-5b6282bcd415 None None] MFA Rules not processed for user `e8ee2ac0e2de411abec414e3dee9bedb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:19.638280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-63b799c6-aac2-4fe3-9f48-5b6282bcd415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:19.709886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1261/2511] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:19.721225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:19.723245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:19.800324 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13ef99-5f8a-4c89-80d5-a05f1bc0beff admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:19.805714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:19.806363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:19.806647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:19.806919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:19.812964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:19.815515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:19.817376 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:19.828210 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1251/2512] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:21.143442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:21.143936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:21.144058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:21.163568 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:21.164793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:21.231158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1262/2513] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:47:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:21.278654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:21.280031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:21.356661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbc8e1-b782-4614-a160-336c22c380b1 service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:21.361705 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f736a-9cb1-4850-b143-c444492819f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:21.362383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f736a-9cb1-4850-b143-c444492819f3 service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:21.363025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f736a-9cb1-4850-b143-c444492819f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:21.363456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f736a-9cb1-4850-b143-c444492819f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:21.372212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f736a-9cb1-4850-b143-c444492819f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:21.373805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f736a-9cb1-4850-b143-c444492819f3 service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:21.382423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1252/2514] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:47:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:47:22.542066 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:22.626507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0445452-2ca8-40f9-937c-dbe075375550 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:22.632048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:22.632806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:22.633200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:22.633582 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] PATH_INFO: `/v3/users/e8ee2ac0e2de411abec414e3dee9bedb` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:22.634968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ee2ac0e2de411abec414e3dee9bedb)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:22.637346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:22.640539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:22.706947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ee2ac0e2de411abec414e3dee9bedb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:22.715259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ee2ac0e2de411abec414e3dee9bedb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:22.715958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ee2ac0e2de411abec414e3dee9bedb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:22.723951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ee2ac0e2de411abec414e3dee9bedb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:22.738197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ee2ac0e2de411abec414e3dee9bedb'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:22.746267 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1263/2515] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:22 2022] DELETE /identity/v3/users/e8ee2ac0e2de411abec414e3dee9bedb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:22.750795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:22.751764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:22.752211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:22.783690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:22.785806 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:23.003794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1253/2516] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:23.012254 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:23.014094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:23.116002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510284b5-9853-46a5-bf11-100758bb48e4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:23.120007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9592c4-5ca1-438b-ab49-c55f228fa151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:23.120631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9592c4-5ca1-438b-ab49-c55f228fa151 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:23.120903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9592c4-5ca1-438b-ab49-c55f228fa151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:23.121167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9592c4-5ca1-438b-ab49-c55f228fa151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:23.140837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9592c4-5ca1-438b-ab49-c55f228fa151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:23.142555 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9592c4-5ca1-438b-ab49-c55f228fa151 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:23.169048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1264/2517] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:23.484248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:23.485970 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:23.520031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dec1c-b39f-41fa-9391-f0ce3d42b43d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:23.527916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:23.528577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:23.529069 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:23.529381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] PATH_INFO: `/v3/projects/96979d19de8a4c479a6538a6b35f8798` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:23.530622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96979d19de8a4c479a6538a6b35f8798)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:23.532979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:23.602411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96979d19de8a4c479a6538a6b35f8798', 'request_id': 'req-e48f7e39-7790-4c56-8339-46c30265ded6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:23.609259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96979d19de8a4c479a6538a6b35f8798', 'request_id': 'req-e48f7e39-7790-4c56-8339-46c30265ded6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:23.622936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96979d19de8a4c479a6538a6b35f8798', 'request_id': 'req-e48f7e39-7790-4c56-8339-46c30265ded6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:23.650061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] The token cache is being invalidate because project 96979d19de8a4c479a6538a6b35f8798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:23.650656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:23.651710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e48f7e39-7790-4c56-8339-46c30265ded6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:23.653761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1254/2518] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:23 2022] DELETE /identity/v3/projects/96979d19de8a4c479a6538a6b35f8798 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:23.659377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:23.659843 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:23.659954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:23.680964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:23.682694 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:23.775472 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1265/2519] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:23.780599 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:23.781394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:23.781394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:23.816921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:23.819269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:23.833902 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1255/2520] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:23.835313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:23.847480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1aab0-d3d7-42e4-bf40-c555918c06b1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:23.852506 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:23.853132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:23.853411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:23.853682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:23.854380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:23.854693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:23.856314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:23.870033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1266/2521] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:23.874361 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:23.882534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:23.887189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb194-81fb-49f5-9e19-2a60846008c6 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:23.896407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:23.897448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:23.897985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:23.898594 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:23.899663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:23.902389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:23.944375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1256/2522] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:23 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:23.948387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:23.963103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5563b-5e5c-4fd5-98f0-43fe286aa8e6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:23.974452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:24.041908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:24.043586 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:24.043868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:24.044138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:24.044787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:24.046771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:24.070578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1267/2523] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:23 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:47:24.074676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:24.086287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43be0a0-c81c-4689-aedb-13227cb08e89 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:24.092543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:24.093192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:24.093587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:24.093862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:24.094560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:24.096209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:24.107134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1257/2524] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:24.111420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:24.123262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49bbf4-cb73-4e0e-b81d-d86bf785a114 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:24.129932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ac7a2-eb4b-434c-a883-b013d5a088a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:24.130699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ac7a2-eb4b-434c-a883-b013d5a088a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:24.130987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ac7a2-eb4b-434c-a883-b013d5a088a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:24.131261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ac7a2-eb4b-434c-a883-b013d5a088a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:24.131926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ac7a2-eb4b-434c-a883-b013d5a088a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:24.133618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ac7a2-eb4b-434c-a883-b013d5a088a1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:24.149039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1268/2525] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:24.150692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:24.161604 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b441c1-994e-4183-a1e2-ad55cca3ecaf admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:24.167049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:24.167837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:24.167837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:24.168050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] PATH_INFO: `/v3/projects/abed47743fb84fa3a6fc82e4775ef95e/users/d2911556ef9c4a15856877e37b07aabe/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:24.184663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abed47743fb84fa3a6fc82e4775ef95e, user_id=d2911556ef9c4a15856877e37b07aabe, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:24.188072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:24.203562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1258/2526] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:24 2022] PUT /identity/v3/projects/abed47743fb84fa3a6fc82e4775ef95e/users/d2911556ef9c4a15856877e37b07aabe/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:24.209507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717361e-1ec5-4616-bdb0-5bf2e941c9e2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:24.209880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717361e-1ec5-4616-bdb0-5bf2e941c9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:24.210246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717361e-1ec5-4616-bdb0-5bf2e941c9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:24.231539 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0717361e-1ec5-4616-bdb0-5bf2e941c9e2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:24.232800 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0717361e-1ec5-4616-bdb0-5bf2e941c9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:24.312994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1269/2527] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:24.328663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:24.330709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:24.435606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc7ed6-c01b-49ee-aa81-62cc4b21cb35 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:24.439802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:24.440482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:24.440765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:24.441039 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:24.452723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:24.454428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:24.468690 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1259/2528] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:25.211580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.212461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.212733 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.250222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] MFA Rules not processed for user `d2911556ef9c4a15856877e37b07aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:25.251525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:25.428121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1270/2529] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:25 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:25.429078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:25.441230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5ed6f-f9ed-4f5e-8657-05dfe33765fe service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:25.447396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:25.448035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.448325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.449152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.449762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:25.452384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:25.499116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1260/2530] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:25 2022] POST /identity/v3/projects => generated 405 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:25.502957 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:25.525720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99285d-7da8-45cd-9bd4-5d50e6afdbb4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:25.562379 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:25.630592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:25.631469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.631829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.632165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.632942 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:25.635433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:25.670187 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1271/2531] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:25 2022] POST /identity/v3/users => generated 309 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:25.672332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:25.690617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa1703-c1aa-4fea-a971-a26e7bd50805 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:25.701978 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:25.702568 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.702846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.703116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.703761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:25.705432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:25.733731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1261/2532] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:25.737256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:25.748332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c926-d8f2-411a-970c-b6f1949a0b1b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:25.755965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:25.755965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.755965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.755965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] PATH_INFO: `/v3/projects/e33864a1854443ce84ec847f3e3d8df3/users/4453a52a741c48feace7c4926d2582a6/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.776383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33864a1854443ce84ec847f3e3d8df3, user_id=4453a52a741c48feace7c4926d2582a6, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:25.779541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:25.798431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1272/2533] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:25 2022] PUT /identity/v3/projects/e33864a1854443ce84ec847f3e3d8df3/users/4453a52a741c48feace7c4926d2582a6/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:25.802298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:25.813198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6bac1-9598-4a32-b1b5-c7f655145299 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:25.879236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41692f1d-deb0-4df2-87b5-5cae175f3193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:25.879969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41692f1d-deb0-4df2-87b5-5cae175f3193 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.880306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41692f1d-deb0-4df2-87b5-5cae175f3193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.880637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41692f1d-deb0-4df2-87b5-5cae175f3193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.881360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41692f1d-deb0-4df2-87b5-5cae175f3193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:25.883044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41692f1d-deb0-4df2-87b5-5cae175f3193 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:25.893624 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1262/2534] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:25.897953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:25.910628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe5630-88d1-408d-8909-4896eae836db admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:25.918766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:25.919471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:25.919806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:25.920084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] PATH_INFO: `/v3/projects/e33864a1854443ce84ec847f3e3d8df3/users/4453a52a741c48feace7c4926d2582a6/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:25.922168 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33864a1854443ce84ec847f3e3d8df3, user_id=4453a52a741c48feace7c4926d2582a6, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:25.924868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:25.936706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1273/2535] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:25 2022] PUT /identity/v3/projects/e33864a1854443ce84ec847f3e3d8df3/users/4453a52a741c48feace7c4926d2582a6/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:26.706463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:26.707020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:26.707149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:26.739051 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:26.751156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] MFA Rules not processed for user `4453a52a741c48feace7c4926d2582a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:26.753112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:26.908569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1263/2536] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:26 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:28.333979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:28.422489 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ef47a-9467-46e2-a55d-7a9c14dbc247 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:28.433949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:28.434780 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:28.435136 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:28.435474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] PATH_INFO: `/v3/users/d2911556ef9c4a15856877e37b07aabe` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:28.436703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2911556ef9c4a15856877e37b07aabe)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:28.438802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:28.503570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2911556ef9c4a15856877e37b07aabe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.512932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2911556ef9c4a15856877e37b07aabe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.513647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2911556ef9c4a15856877e37b07aabe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.524484 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2911556ef9c4a15856877e37b07aabe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.533757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:28.539339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2911556ef9c4a15856877e37b07aabe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.550572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1274/2537] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:28 2022] DELETE /identity/v3/users/d2911556ef9c4a15856877e37b07aabe => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:28.749209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:28.765623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7c9e9-50e7-4044-af7f-e97e5cddc03c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:28.848525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:28.849567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:28.849885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:28.850189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] PATH_INFO: `/v3/projects/abed47743fb84fa3a6fc82e4775ef95e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:28.851467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] RBAC: Authorizing `identity:delete_project(project_id=abed47743fb84fa3a6fc82e4775ef95e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:28.853383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:28.911168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abed47743fb84fa3a6fc82e4775ef95e', 'request_id': 'req-af0ca5ae-73aa-4175-8599-87a3ac72305d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.917931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abed47743fb84fa3a6fc82e4775ef95e', 'request_id': 'req-af0ca5ae-73aa-4175-8599-87a3ac72305d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.925150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abed47743fb84fa3a6fc82e4775ef95e', 'request_id': 'req-af0ca5ae-73aa-4175-8599-87a3ac72305d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.941821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] The token cache is being invalidate because project abed47743fb84fa3a6fc82e4775ef95e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:28.942022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.942680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:28.944182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1264/2538] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:28 2022] DELETE /identity/v3/projects/abed47743fb84fa3a6fc82e4775ef95e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:28.948375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:28.949921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:28.966119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4a300-0bf5-436c-956c-3493329d377a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:29.027400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:29.028229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:29.028577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:29.028937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] PATH_INFO: `/v3/users/4453a52a741c48feace7c4926d2582a6` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:29.032352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=4453a52a741c48feace7c4926d2582a6)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:29.032518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:29.088857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4453a52a741c48feace7c4926d2582a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.098724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4453a52a741c48feace7c4926d2582a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.099661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4453a52a741c48feace7c4926d2582a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.109677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4453a52a741c48feace7c4926d2582a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.124846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2f9e39c4-f86b-4903-8b9d-837214e191ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4453a52a741c48feace7c4926d2582a6'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.133840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1275/2539] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:28 2022] DELETE /identity/v3/users/4453a52a741c48feace7c4926d2582a6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:29.353329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:29.355807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:29.374837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca5ae-73aa-4175-8599-87a3ac72305d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:29.420906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:29.510286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:29.512019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:29.512297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:29.512569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] PATH_INFO: `/v3/projects/e33864a1854443ce84ec847f3e3d8df3` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:29.514299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33864a1854443ce84ec847f3e3d8df3)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:29.516669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:29.627771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33864a1854443ce84ec847f3e3d8df3', 'request_id': 'req-56c3e363-5b63-4966-a0be-422b51db660d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.637031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33864a1854443ce84ec847f3e3d8df3', 'request_id': 'req-56c3e363-5b63-4966-a0be-422b51db660d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.644353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33864a1854443ce84ec847f3e3d8df3', 'request_id': 'req-56c3e363-5b63-4966-a0be-422b51db660d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.678499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] The token cache is being invalidate because project e33864a1854443ce84ec847f3e3d8df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:29.678828 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.679612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-56c3e363-5b63-4966-a0be-422b51db660d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:29.681468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1265/2540] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:29 2022] DELETE /identity/v3/projects/e33864a1854443ce84ec847f3e3d8df3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:29.686714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:29.687270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:29.687467 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:29.710921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:29.712236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:29.802207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1276/2541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:29.810622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:29.811178 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:29.811363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:29.835375 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:29.837329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:29.845818 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1266/2542] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:29.850845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:29.863424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedc4d-096f-4216-88db-13d9d99e4bf7 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:29.868440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:29.869148 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:29.869494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:29.869932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:29.870700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:29.871217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:29.872897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:29.881097 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:29.888255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1277/2543] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:29.892774 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:29.909233 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b4703-e873-4e58-84a4-e92407e210a8 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:29.915435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:29.916136 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:29.916489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:29.916829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:29.917535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:29.920936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:29.951402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1267/2544] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:29 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:29.955520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:29.966897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae096-bae9-4c65-947a-31fc6a8b7f31 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.026590 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.027612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.028107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.028444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.029153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.031193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.057983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1278/2545] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:29 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:47:30.062115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.073318 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522ff69-1f9c-4a4f-979a-fec15d2152a2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.078125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.080862 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.081196 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.081468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.082116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.083895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.097138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1268/2546] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:30.101391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.112478 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afedbc-d316-465c-90ab-c793ba587673 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.117775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522afe-f47b-43ee-b4cb-2019fd8885c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.118426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522afe-f47b-43ee-b4cb-2019fd8885c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.118708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522afe-f47b-43ee-b4cb-2019fd8885c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.118976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522afe-f47b-43ee-b4cb-2019fd8885c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.119628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522afe-f47b-43ee-b4cb-2019fd8885c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.122063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522afe-f47b-43ee-b4cb-2019fd8885c2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.135392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1279/2547] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:30.139501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.151312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d21a-1e3b-4ea9-9383-d64b6c006657 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.158098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.159461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.160001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.160553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] PATH_INFO: `/v3/projects/919d8a1cadec4b8aa07629c8bdd915a2/users/7417d3599bb34c8f9ec989b7995b38af/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.181101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=919d8a1cadec4b8aa07629c8bdd915a2, user_id=7417d3599bb34c8f9ec989b7995b38af, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.183895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.203462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1269/2548] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:30 2022] PUT /identity/v3/projects/919d8a1cadec4b8aa07629c8bdd915a2/users/7417d3599bb34c8f9ec989b7995b38af/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:30.213866 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.214450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.214888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.247857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] MFA Rules not processed for user `7417d3599bb34c8f9ec989b7995b38af`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:30.249133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:30.327280 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1280/2549] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:30.331985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.390474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:30.405963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e835948-36e4-43bf-abfe-2e5274ad6fe7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.412211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.412852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.413131 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.413406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.414056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.418192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.453850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1270/2550] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:30 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:30.458251 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.469874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a243f1-cf71-42ca-9421-2f5dab119b2c None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.535032 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.535686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.535952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.536242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.536932 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.539025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.550620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:30.563140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1281/2551] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:30 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:47:30.567010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.593762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1abcea-88d9-4473-8ddf-e08a42eea521 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.609215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.609215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.609215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.609215 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.610315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.610315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.618553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1271/2552] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:30.619605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.631007 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef36e0-639c-49ed-887e-c053ae4b15df admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.637201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.637884 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.638200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.638546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] PATH_INFO: `/v3/projects/5912e74240224b8c8c9d362d8f89aa86/users/46acc45023fe4f4da38dfb1c0838874f/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.670067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5912e74240224b8c8c9d362d8f89aa86, user_id=46acc45023fe4f4da38dfb1c0838874f, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.672645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.701581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1282/2553] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:30 2022] PUT /identity/v3/projects/5912e74240224b8c8c9d362d8f89aa86/users/46acc45023fe4f4da38dfb1c0838874f/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:30.703145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.716538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f702c9-5705-4d84-9aa8-398c6f5b267d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.789304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ec93d-a599-4a7e-90ca-b71435a03ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.790041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec93d-a599-4a7e-90ca-b71435a03ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.790452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec93d-a599-4a7e-90ca-b71435a03ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.790825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec93d-a599-4a7e-90ca-b71435a03ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.791577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec93d-a599-4a7e-90ca-b71435a03ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.793299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec93d-a599-4a7e-90ca-b71435a03ddd admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.811292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1272/2554] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:30.815367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:30.826693 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266f04e-5db8-4c45-8763-fd803e895b95 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:30.832494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:30.832494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.832494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.832868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] PATH_INFO: `/v3/projects/5912e74240224b8c8c9d362d8f89aa86/users/46acc45023fe4f4da38dfb1c0838874f/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.834764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5912e74240224b8c8c9d362d8f89aa86, user_id=46acc45023fe4f4da38dfb1c0838874f, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:30.837470 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:30.864953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1283/2555] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:30 2022] PUT /identity/v3/projects/5912e74240224b8c8c9d362d8f89aa86/users/46acc45023fe4f4da38dfb1c0838874f/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:30.866427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:30.866672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:30.866998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:30.900727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] MFA Rules not processed for user `46acc45023fe4f4da38dfb1c0838874f`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:30.903857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:30.988561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1273/2556] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:31.007982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:31.010939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:31.093373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff2f6b-a550-4888-b676-50bf8c09b3e6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:31.101362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:31.102006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:31.102320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:31.102594 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:31.112587 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:31.114543 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:31.128442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1284/2557] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:31.608821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:31.619212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80657b03-9f2e-4e63-adcf-3e7f11ac103a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:31.624282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46acc45023fe4f4da38dfb1c0838874f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5912e74240224b8c8c9d362d8f89aa86', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:31.625771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:31.626097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:31.626407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:31.627074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:31.627381 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:31.628991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:31.648218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1274/2558] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:31.648462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:31.661963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92769aa5-c521-4fce-ba03-20a3ec430290 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:31.668325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46acc45023fe4f4da38dfb1c0838874f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5912e74240224b8c8c9d362d8f89aa86', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:31.668989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:31.669289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:31.669569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:31.670436 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:31.672505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:31.704165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1285/2559] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:31 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:47:31.708169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:31.722064 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0090d16-b82d-43b2-a792-37d508e64f6b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:31.760382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:31.786942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46acc45023fe4f4da38dfb1c0838874f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5912e74240224b8c8c9d362d8f89aa86', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:31.787595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:31.787880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:31.788150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:31.788794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:31.789095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:31.790715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:31.801744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1275/2560] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:31.841923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:31.853114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c9be1-5b73-4bb8-a907-30ee1b0c67ee tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:31.857822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46acc45023fe4f4da38dfb1c0838874f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5912e74240224b8c8c9d362d8f89aa86', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:31.858485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:31.858754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:31.859023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] PATH_INFO: `/v3/projects/703dfd6481744a12af025cfdb463ca4b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:31.860153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=703dfd6481744a12af025cfdb463ca4b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:31.862088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:31.911591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703dfd6481744a12af025cfdb463ca4b', 'request_id': 'req-17257b6f-2485-4988-8a0e-e5882a926b40'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:31.920063 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703dfd6481744a12af025cfdb463ca4b', 'request_id': 'req-17257b6f-2485-4988-8a0e-e5882a926b40'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:31.927040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703dfd6481744a12af025cfdb463ca4b', 'request_id': 'req-17257b6f-2485-4988-8a0e-e5882a926b40'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:31.938429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] The token cache is being invalidate because project 703dfd6481744a12af025cfdb463ca4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:31.938818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:31.939583 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-17257b6f-2485-4988-8a0e-e5882a926b40 tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:31.941192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1286/2561] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:31 2022] DELETE /identity/v3/projects/703dfd6481744a12af025cfdb463ca4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:32.143262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:32.145250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:32.210078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a74469-5475-43e4-ad02-a31919c5028b tempest-VolumeQuotaClassesTest-1905834939 tempest-VolumeQuotaClassesTest-1905834939-project-admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:32.267111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:32.267764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:32.268045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:32.268342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] PATH_INFO: `/v3/users/7417d3599bb34c8f9ec989b7995b38af` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:32.269753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7417d3599bb34c8f9ec989b7995b38af)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:32.272078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:32.320056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7417d3599bb34c8f9ec989b7995b38af'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.320698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7417d3599bb34c8f9ec989b7995b38af'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.329370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7417d3599bb34c8f9ec989b7995b38af'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.336885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7417d3599bb34c8f9ec989b7995b38af'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.348936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:32.350500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7417d3599bb34c8f9ec989b7995b38af'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.358008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1276/2562] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:32 2022] DELETE /identity/v3/users/7417d3599bb34c8f9ec989b7995b38af => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:32.363679 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:32.364043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:32.364422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:32.389228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:32.390492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:32.408552 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:32.465511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1287/2563] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:32.475240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:32.476856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:32.542670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a145627-15d0-4046-8fed-ee37087ce352 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:32.547633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:32.548321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:32.548634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:32.548906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:32.559177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:32.560873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:32.571720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1277/2564] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:47:32.861534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:32.864114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:32.880236 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73f091-ee22-4e88-88c8-6bac240a3c8d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:32.887468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:32.888027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:32.888306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:32.888580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] PATH_INFO: `/v3/projects/919d8a1cadec4b8aa07629c8bdd915a2` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:32.889925 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=919d8a1cadec4b8aa07629c8bdd915a2)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:32.891997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:32.940329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919d8a1cadec4b8aa07629c8bdd915a2', 'request_id': 'req-770f3f59-6ece-482f-93fe-895339722eba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.947795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919d8a1cadec4b8aa07629c8bdd915a2', 'request_id': 'req-770f3f59-6ece-482f-93fe-895339722eba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.954016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919d8a1cadec4b8aa07629c8bdd915a2', 'request_id': 'req-770f3f59-6ece-482f-93fe-895339722eba'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.964291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] The token cache is being invalidate because project 919d8a1cadec4b8aa07629c8bdd915a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:32.964579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.965208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:32.966750 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1288/2565] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:32 2022] DELETE /identity/v3/projects/919d8a1cadec4b8aa07629c8bdd915a2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:32.970861 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:32.980817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c056d-6cbd-4608-8f4e-2f5c59471c14 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.037741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.038486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.038868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.039120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] PATH_INFO: `/v3/users/46acc45023fe4f4da38dfb1c0838874f` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.040435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] RBAC: Authorizing `identity:delete_user(user_id=46acc45023fe4f4da38dfb1c0838874f)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.042385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.100450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46acc45023fe4f4da38dfb1c0838874f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.101162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46acc45023fe4f4da38dfb1c0838874f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.110853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46acc45023fe4f4da38dfb1c0838874f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.119794 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46acc45023fe4f4da38dfb1c0838874f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.132567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-eef2689f-0da3-4a2d-bbbe-a30d27e9abed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46acc45023fe4f4da38dfb1c0838874f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.139654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1278/2566] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:32 2022] DELETE /identity/v3/users/46acc45023fe4f4da38dfb1c0838874f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:33.419344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.421034 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:33.434896 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f3f59-6ece-482f-93fe-895339722eba admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.491352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.491911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.492186 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.492455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] PATH_INFO: `/v3/projects/5912e74240224b8c8c9d362d8f89aa86` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.493624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5912e74240224b8c8c9d362d8f89aa86)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.495541 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.541944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:33.543242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5912e74240224b8c8c9d362d8f89aa86', 'request_id': 'req-ae451e88-46a1-42e0-8840-3e9d18744f03'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.553565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5912e74240224b8c8c9d362d8f89aa86', 'request_id': 'req-ae451e88-46a1-42e0-8840-3e9d18744f03'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.560315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5912e74240224b8c8c9d362d8f89aa86', 'request_id': 'req-ae451e88-46a1-42e0-8840-3e9d18744f03'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.572372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] The token cache is being invalidate because project 5912e74240224b8c8c9d362d8f89aa86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:33.572659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.573316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-ae451e88-46a1-42e0-8840-3e9d18744f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:33.574898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1289/2567] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:33 2022] DELETE /identity/v3/projects/5912e74240224b8c8c9d362d8f89aa86 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:33.579960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.580446 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.580543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.606699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:33.608065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:33.620906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:33.676135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1279/2568] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:33.684356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.684676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.684947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.706683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:33.707939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:33.716919 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1290/2569] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:33.721310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.731405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe3a45-879a-47c4-8947-faef03201491 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.736839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.737443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.737717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.738382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.739032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:33.739335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.740937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.752879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1280/2570] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:33.757192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.767660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d8ef7-e351-4fa0-b4a2-87aa53cf5f4f None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.774482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.775095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.775367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.775645 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.776295 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.778244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.808240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1291/2571] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:33 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:33.812572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.822854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592420b-3af4-4728-ab41-e1694524ea50 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.879495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.880102 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.880383 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.880653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.881282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.883258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.909750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1281/2572] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:33 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:33.914038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.924782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857e63-402c-4896-8e25-e337a62db4dd admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.929944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.930573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.930849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.931114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.931789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.933352 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.943400 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1292/2573] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:33.947587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.958019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a49ea1-9fc3-4332-b644-42487fb4592e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.963014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cf398-5196-40a6-85cb-5eeb25d10d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.963611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cf398-5196-40a6-85cb-5eeb25d10d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.963888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cf398-5196-40a6-85cb-5eeb25d10d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.964157 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cf398-5196-40a6-85cb-5eeb25d10d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:33.964799 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cf398-5196-40a6-85cb-5eeb25d10d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:33.966415 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cf398-5196-40a6-85cb-5eeb25d10d44 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:33.977331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1282/2574] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:33.981475 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:33.991968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699168-0388-4e0e-a2ef-31f9cb49d6e4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:33.997077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:33.997716 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:33.997989 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:33.998290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] PATH_INFO: `/v3/projects/622094c9b88f4d3faeca3d53b52bc37c/users/c94522112aae4488b12f6fc4a5b41e64/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:34.012640 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=622094c9b88f4d3faeca3d53b52bc37c, user_id=c94522112aae4488b12f6fc4a5b41e64, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:34.015264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:34.030906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1293/2575] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:33 2022] PUT /identity/v3/projects/622094c9b88f4d3faeca3d53b52bc37c/users/c94522112aae4488b12f6fc4a5b41e64/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:34.042207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:34.042639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:34.043089 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:34.074315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] MFA Rules not processed for user `c94522112aae4488b12f6fc4a5b41e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:34.075577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:34.144174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1283/2576] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:34.155723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:34.157747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:34.213994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00b83-b0b0-4f4b-a0a9-6e3e775b1428 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:34.217920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a8c05-861f-456a-8241-d706a8d266e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:34.218629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a8c05-861f-456a-8241-d706a8d266e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:34.218901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a8c05-861f-456a-8241-d706a8d266e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:34.219164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a8c05-861f-456a-8241-d706a8d266e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:34.227876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a8c05-861f-456a-8241-d706a8d266e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:34.229298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a8c05-861f-456a-8241-d706a8d266e1 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:34.238483 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1294/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:37.460311 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:37.477114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:37.536360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada0c6-a554-4d6b-868f-8a33f1dc0ef4 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:37.541698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:37.542359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:37.542760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:37.543235 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] PATH_INFO: `/v3/users/c94522112aae4488b12f6fc4a5b41e64` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:37.544426 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94522112aae4488b12f6fc4a5b41e64)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:37.546358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:37.595340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94522112aae4488b12f6fc4a5b41e64'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:37.596006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94522112aae4488b12f6fc4a5b41e64'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:37.604982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94522112aae4488b12f6fc4a5b41e64'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:37.611741 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94522112aae4488b12f6fc4a5b41e64'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:37.623398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94522112aae4488b12f6fc4a5b41e64'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:37.629744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1284/2578] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:37 2022] DELETE /identity/v3/users/c94522112aae4488b12f6fc4a5b41e64 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:37.634521 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:37.635002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:37.635130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:37.654089 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:37.655330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:37.718984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1295/2579] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:47:37.729165 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:37.730726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:37.792830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973edb43-a233-4cc4-9371-eed71342dd98 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:37.796782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da4d5c-e792-4ebb-b62f-8aaf23cbd3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:37.797397 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da4d5c-e792-4ebb-b62f-8aaf23cbd3ec service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:37.797678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da4d5c-e792-4ebb-b62f-8aaf23cbd3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:37.797949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da4d5c-e792-4ebb-b62f-8aaf23cbd3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:37.808411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da4d5c-e792-4ebb-b62f-8aaf23cbd3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:37.809874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da4d5c-e792-4ebb-b62f-8aaf23cbd3ec service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:37.820113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1285/2580] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:38.095477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.097192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:38.110073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe5f8c-f983-4cee-908d-a6f0739dc92d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.115659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.116303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.116712 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.116880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] PATH_INFO: `/v3/projects/622094c9b88f4d3faeca3d53b52bc37c` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.118040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=622094c9b88f4d3faeca3d53b52bc37c)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.120130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.132502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:38.171260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622094c9b88f4d3faeca3d53b52bc37c', 'request_id': 'req-c82501b4-55de-41d8-80c9-8be770a4d31b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:38.179027 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622094c9b88f4d3faeca3d53b52bc37c', 'request_id': 'req-c82501b4-55de-41d8-80c9-8be770a4d31b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:38.185356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622094c9b88f4d3faeca3d53b52bc37c', 'request_id': 'req-c82501b4-55de-41d8-80c9-8be770a4d31b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:38.196242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] The token cache is being invalidate because project 622094c9b88f4d3faeca3d53b52bc37c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:38.196540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:38.197226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c82501b4-55de-41d8-80c9-8be770a4d31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:38.198854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1296/2581] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:38 2022] DELETE /identity/v3/projects/622094c9b88f4d3faeca3d53b52bc37c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:38.203939 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.204236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.204517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.223860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:38.225056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:38.255907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:38.289853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1286/2582] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:38.299221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.299509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.299781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.320752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:38.321962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:38.331178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1297/2583] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:38.337625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.348044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be73fe7-7f61-4c61-9316-2bb9c0f753b3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.355761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.356676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.357052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.357336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.358112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:38.358561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.361042 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.374248 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1287/2584] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:38.378495 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.387823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f523f63-9547-48a1-aec2-4a650cb6c942 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.392986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.393577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.393848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.394120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.394789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.396685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.427463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1298/2585] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:38.431813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.442716 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968eb8-73ec-4a75-9149-63d528082c88 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.498617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.499276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.499475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.499748 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.500356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.502305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.526494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1288/2586] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:38.530671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.540912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1befc-bdf1-488b-96eb-1ca926d8cda1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.546269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.546876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.547150 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.547414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.548161 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.549697 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.559741 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1299/2587] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:38.563735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.574831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cc6a2-4782-4ced-8bb4-585865bf31ee admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.579839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53723c93-9944-4ac7-b394-251445d149a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.580492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53723c93-9944-4ac7-b394-251445d149a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.580752 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53723c93-9944-4ac7-b394-251445d149a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.581022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53723c93-9944-4ac7-b394-251445d149a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.581680 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53723c93-9944-4ac7-b394-251445d149a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.583275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53723c93-9944-4ac7-b394-251445d149a9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.593869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1289/2588] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:38.598344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.608938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a2dc-da23-46c6-9218-880781e1f9a6 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.614823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.615480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.615763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.616036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] PATH_INFO: `/v3/projects/1d9a88f2f12249de9c9c988a01ee0d1c/users/67553b837e9f4a56b3f165c623798706/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.630831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9a88f2f12249de9c9c988a01ee0d1c, user_id=67553b837e9f4a56b3f165c623798706, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.633425 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.648649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1300/2589] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:38 2022] PUT /identity/v3/projects/1d9a88f2f12249de9c9c988a01ee0d1c/users/67553b837e9f4a56b3f165c623798706/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:38.659569 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.659855 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.660130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.688809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] MFA Rules not processed for user `67553b837e9f4a56b3f165c623798706`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:38.690138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:38.760685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1290/2590] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:38.765402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.832883 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06911c9d-46c2-4e36-89dc-599e7b705ba8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.838700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.839311 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.839618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.839892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.840528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.842455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.870081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1301/2591] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:38.876226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.888027 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc78de-2f10-42b2-8a7c-d50225f64764 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.918964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:38.950883 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.951506 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.951808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.952083 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.952709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:38.954703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:38.976247 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1291/2592] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:38 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:38.980462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:38.991553 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ea6a-c76f-40e2-b9f8-4d9a3ef5a154 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:38.996706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:38.997299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:38.997569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:38.997842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:38.998531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.000128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.010816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1302/2593] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:39.014880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.025438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32455a83-31d3-4875-b4af-79917d67235f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.030367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776f12-2b7f-47aa-8192-e6c7c62b98b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.030992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776f12-2b7f-47aa-8192-e6c7c62b98b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.031275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776f12-2b7f-47aa-8192-e6c7c62b98b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.031547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776f12-2b7f-47aa-8192-e6c7c62b98b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.032195 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776f12-2b7f-47aa-8192-e6c7c62b98b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.033805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776f12-2b7f-47aa-8192-e6c7c62b98b0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.044197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1292/2594] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:39.048446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.059031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73c3a0-cc61-4308-8dfd-0adb9c90c92a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.064281 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.065105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.065387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.065660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] PATH_INFO: `/v3/projects/e67f2efc9427447db7d86d1332a6ef62/users/f040d8b8819541149fe447ba7aff46fd/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.068965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:39.084068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67f2efc9427447db7d86d1332a6ef62, user_id=f040d8b8819541149fe447ba7aff46fd, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.087297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.105006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1303/2595] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:39 2022] PUT /identity/v3/projects/e67f2efc9427447db7d86d1332a6ef62/users/f040d8b8819541149fe447ba7aff46fd/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:39.123168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.123528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.123897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.160282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] MFA Rules not processed for user `f040d8b8819541149fe447ba7aff46fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:39.161451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:39.228067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1293/2596] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:39.232623 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.296748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1434d-55b0-4e32-a75e-73d23b5a45ba admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.302245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.302703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.302981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.303246 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.303902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.305820 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.333803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1304/2597] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:39 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:39.337875 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.348936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71280e4-4fc1-427e-82b8-fecc6094ec6a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.406625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.407240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.407539 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.408270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.408995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.411013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.437601 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1294/2598] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:39 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:39.442077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.452808 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4aac4-b69f-4b51-b4dc-55102db834c7 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.457891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.458508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.458790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.459057 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.459732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.461343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.471816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1305/2599] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:39.476035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.486475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd4747-b816-4c38-a20f-1b853406c82b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.491937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.492589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.492845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.493118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] PATH_INFO: `/v3/projects/9199794097ab4fa6827e7a82ed423bcd/users/55d36cf8ffa149bfb61365e7c944f3b7/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.508314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9199794097ab4fa6827e7a82ed423bcd, user_id=55d36cf8ffa149bfb61365e7c944f3b7, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.511044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.526517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1295/2600] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:39 2022] PUT /identity/v3/projects/9199794097ab4fa6827e7a82ed423bcd/users/55d36cf8ffa149bfb61365e7c944f3b7/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:39.530809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.541176 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742ede3-b178-48d0-b1c6-b6c1c3db3701 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.603526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d696791-f3a2-49f9-8267-f70d30b4e8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.604146 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d696791-f3a2-49f9-8267-f70d30b4e8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.604421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d696791-f3a2-49f9-8267-f70d30b4e8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.604692 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d696791-f3a2-49f9-8267-f70d30b4e8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.605347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d696791-f3a2-49f9-8267-f70d30b4e8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.606984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d696791-f3a2-49f9-8267-f70d30b4e8ef admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.617423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1306/2601] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:39.621635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.628857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:39.632919 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846f86-d99c-46ad-8607-0ebedb01341f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.638703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.639358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.639638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.639911 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] PATH_INFO: `/v3/projects/9199794097ab4fa6827e7a82ed423bcd/users/55d36cf8ffa149bfb61365e7c944f3b7/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.641934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9199794097ab4fa6827e7a82ed423bcd, user_id=55d36cf8ffa149bfb61365e7c944f3b7, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.644581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.654988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1296/2602] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:39 2022] PUT /identity/v3/projects/9199794097ab4fa6827e7a82ed423bcd/users/55d36cf8ffa149bfb61365e7c944f3b7/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:39.666569 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.667076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.667076 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.696428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] MFA Rules not processed for user `55d36cf8ffa149bfb61365e7c944f3b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:39.697660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:39.755952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:39.782759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1307/2603] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:39.794417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:39.796495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:39.856892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4975d-7513-4385-9ea7-38c8b5ad5d5d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:39.861800 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dfe39-752d-4dbf-9627-66390ee1ca82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:39.862430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dfe39-752d-4dbf-9627-66390ee1ca82 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:39.862689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dfe39-752d-4dbf-9627-66390ee1ca82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:39.862953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dfe39-752d-4dbf-9627-66390ee1ca82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:39.925130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dfe39-752d-4dbf-9627-66390ee1ca82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:39.926579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dfe39-752d-4dbf-9627-66390ee1ca82 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:39.936155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1297/2604] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:41.459465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:41.471598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed259632-3882-421b-a3b5-aa720e4e99bd None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:41.475619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:41.476221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:41.476496 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:41.476764 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:41.534322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:41.535688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:41.544580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1308/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:43.668727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:43.669214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:43.669329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:43.669835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:43.678844 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81468a0b-b18f-4de8-ad8d-68a8b012a373 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:43.682424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:43.683026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:43.683301 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:43.683567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:43.689651 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:43.690986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:43.695036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:43.697370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:43.709647 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1309/2606] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:47:43.767625 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1298/2607] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:47:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:43.780627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:43.782056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:43.862547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80d573-e40a-4a2c-aa63-2478a4e0757a service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:43.868310 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920803d5-8278-4c87-9c05-d3fdda9bb51e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:43.868926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920803d5-8278-4c87-9c05-d3fdda9bb51e service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:43.869204 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920803d5-8278-4c87-9c05-d3fdda9bb51e service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:43.869474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920803d5-8278-4c87-9c05-d3fdda9bb51e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:43.877948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920803d5-8278-4c87-9c05-d3fdda9bb51e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:43.879845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920803d5-8278-4c87-9c05-d3fdda9bb51e service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:43.888508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1310/2608] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:47:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:47:47.745660 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:47.806850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:47.825239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f9d2-3697-4b11-b1ea-6843da7860b3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:47.830852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:47.831653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:47.831996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:47.832340 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] PATH_INFO: `/v3/users/67553b837e9f4a56b3f165c623798706` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:47.833734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=67553b837e9f4a56b3f165c623798706)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:47.835719 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:47.936796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67553b837e9f4a56b3f165c623798706'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:47.938824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67553b837e9f4a56b3f165c623798706'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:47.960986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67553b837e9f4a56b3f165c623798706'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:47.969191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67553b837e9f4a56b3f165c623798706'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:47.994857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67553b837e9f4a56b3f165c623798706'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.002440 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1299/2609] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:47 2022] DELETE /identity/v3/users/67553b837e9f4a56b3f165c623798706 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:48.012124 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:48.012690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:48.012690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:48.038537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:48.039782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:48.105427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:48.124509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1311/2610] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:48.131428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:48.133642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:48.215448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a9bb8-09e5-477e-8340-122e36aebb8f admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:48.220694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:48.221728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:48.222298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:48.222787 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:48.234709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:48.236468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:48.248395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1300/2611] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:48.654559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:48.656460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:48.675939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3d05-bfc9-4403-aae5-b3899441fd8a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:48.692531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:48.694605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:48.694860 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:48.695139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] PATH_INFO: `/v3/projects/1d9a88f2f12249de9c9c988a01ee0d1c` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:48.696461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9a88f2f12249de9c9c988a01ee0d1c)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:48.698409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:48.774875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9a88f2f12249de9c9c988a01ee0d1c', 'request_id': 'req-9c4eb894-f096-4894-b30f-bb9f1614723c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.784581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9a88f2f12249de9c9c988a01ee0d1c', 'request_id': 'req-9c4eb894-f096-4894-b30f-bb9f1614723c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.791264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9a88f2f12249de9c9c988a01ee0d1c', 'request_id': 'req-9c4eb894-f096-4894-b30f-bb9f1614723c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.805247 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] The token cache is being invalidate because project 1d9a88f2f12249de9c9c988a01ee0d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:48.806002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.806945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.809101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1312/2612] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:48 2022] DELETE /identity/v3/projects/1d9a88f2f12249de9c9c988a01ee0d1c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:48.813327 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:48.823613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a16fb-1198-4357-ac7a-e03143c9d3e8 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:48.903546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:48.904481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:48.904782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:48.905061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] PATH_INFO: `/v3/users/f040d8b8819541149fe447ba7aff46fd` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:48.906481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f040d8b8819541149fe447ba7aff46fd)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:48.908475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:48.924696 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:48.985734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f040d8b8819541149fe447ba7aff46fd'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.986630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f040d8b8819541149fe447ba7aff46fd'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:48.996644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f040d8b8819541149fe447ba7aff46fd'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.004536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f040d8b8819541149fe447ba7aff46fd'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.025194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f040d8b8819541149fe447ba7aff46fd'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.032344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1301/2613] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:48 2022] DELETE /identity/v3/users/f040d8b8819541149fe447ba7aff46fd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:49.456494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:49.457969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:49.471570 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eb894-f096-4894-b30f-bb9f1614723c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:49.547358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:49.548226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:49.548515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:49.548842 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] PATH_INFO: `/v3/projects/e67f2efc9427447db7d86d1332a6ef62` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:49.551707 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67f2efc9427447db7d86d1332a6ef62)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:49.553644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:49.603835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67f2efc9427447db7d86d1332a6ef62', 'request_id': 'req-126bc537-d6c4-4358-97f1-72b0634028fe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.614055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67f2efc9427447db7d86d1332a6ef62', 'request_id': 'req-126bc537-d6c4-4358-97f1-72b0634028fe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.622706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67f2efc9427447db7d86d1332a6ef62', 'request_id': 'req-126bc537-d6c4-4358-97f1-72b0634028fe'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.634956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] The token cache is being invalidate because project e67f2efc9427447db7d86d1332a6ef62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:49.635256 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.636177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.641313 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1313/2614] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:49 2022] DELETE /identity/v3/projects/e67f2efc9427447db7d86d1332a6ef62 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:49.642148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:49.643738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:49.663976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fee5f-b80c-458a-9160-9ac70b71dc94 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:49.754576 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:49.756856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:49.757242 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:49.757633 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] PATH_INFO: `/v3/users/55d36cf8ffa149bfb61365e7c944f3b7` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:49.759934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d36cf8ffa149bfb61365e7c944f3b7)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:49.762443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:49.818789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d36cf8ffa149bfb61365e7c944f3b7'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.819491 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d36cf8ffa149bfb61365e7c944f3b7'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.828907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d36cf8ffa149bfb61365e7c944f3b7'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.845598 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d36cf8ffa149bfb61365e7c944f3b7'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.859764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d194b561-2016-428f-abdd-9c3ed4250ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d36cf8ffa149bfb61365e7c944f3b7'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:49.867174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1302/2615] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:49 2022] DELETE /identity/v3/users/55d36cf8ffa149bfb61365e7c944f3b7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:50.211689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.213195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:50.230909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bc537-d6c4-4358-97f1-72b0634028fe admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.242769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:50.311563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.312502 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.312967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.313408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] PATH_INFO: `/v3/projects/9199794097ab4fa6827e7a82ed423bcd` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.314935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9199794097ab4fa6827e7a82ed423bcd)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.316934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.414676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9199794097ab4fa6827e7a82ed423bcd', 'request_id': 'req-27ff393a-c774-44d7-8571-ec726e6bb429'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:50.428471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9199794097ab4fa6827e7a82ed423bcd', 'request_id': 'req-27ff393a-c774-44d7-8571-ec726e6bb429'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:50.446782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9199794097ab4fa6827e7a82ed423bcd', 'request_id': 'req-27ff393a-c774-44d7-8571-ec726e6bb429'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:50.461847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] The token cache is being invalidate because project 9199794097ab4fa6827e7a82ed423bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:50.462476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:50.463228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-27ff393a-c774-44d7-8571-ec726e6bb429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:50.464752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1314/2616] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:50 2022] DELETE /identity/v3/projects/9199794097ab4fa6827e7a82ed423bcd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:50.471204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.471204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.471566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.499511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:50.500920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:50.508559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:50.585190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1303/2617] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:50.593259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.593628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.593975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.616100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:50.617608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:50.627137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1315/2618] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:50.631343 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.641715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657112f-0c53-433e-b30a-a59b12b4e2d1 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.646611 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.647250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.647531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.647801 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.648458 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:50.648760 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.650389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.663360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1304/2619] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:47:50.668234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.678644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70089-810e-447b-87cd-dd4a72b3bda4 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.683687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.684322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.684629 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.684898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.685684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.687723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.719836 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1316/2620] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:50 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:50.720974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.731639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053fc0b-79aa-4e27-91e9-344520b4b872 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.791550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.792085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.792363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.793128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.793250 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.795337 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.819153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1305/2621] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:50 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:50.824303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.839276 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712a6f-93bc-4323-a8fd-7d2d27a1a2ea admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.844626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.845420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.845702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.846031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.846926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.848560 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.864430 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1317/2622] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:50.867471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.883139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93ad11-66c8-4ca3-a357-92e9d18dc8c8 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.888078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff868599-e3bf-4f74-8328-e00e67dba78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.888704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff868599-e3bf-4f74-8328-e00e67dba78c admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.888986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff868599-e3bf-4f74-8328-e00e67dba78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.889262 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff868599-e3bf-4f74-8328-e00e67dba78c admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.889922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff868599-e3bf-4f74-8328-e00e67dba78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.891577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff868599-e3bf-4f74-8328-e00e67dba78c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.902969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1306/2623] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:50.907283 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:50.918323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882ecc0-dfb7-4eac-a51d-b0e7a74c6d72 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:50.923100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:50.923795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:50.924081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:50.924353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] PATH_INFO: `/v3/projects/9b55ea5b09f24be4a71958f780d7a783/users/b7d59ca4a62f41c782a67084fe3db56c/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:50.939603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b55ea5b09f24be4a71958f780d7a783, user_id=b7d59ca4a62f41c782a67084fe3db56c, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:50.942446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:50.958990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1318/2624] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:50 2022] PUT /identity/v3/projects/9b55ea5b09f24be4a71958f780d7a783/users/b7d59ca4a62f41c782a67084fe3db56c/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:51.028618 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:51.028908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:51.029189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:51.057225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] MFA Rules not processed for user `b7d59ca4a62f41c782a67084fe3db56c`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:51.058453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:51.127738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1307/2625] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:51.139839 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:51.141691 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:51.184993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:51.206328 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a96fa-a438-4d20-9153-a70478d22674 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:51.210535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa85623-1bef-4fe0-9c93-de3514999737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:51.211159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85623-1bef-4fe0-9c93-de3514999737 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:51.211442 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85623-1bef-4fe0-9c93-de3514999737 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:51.211713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85623-1bef-4fe0-9c93-de3514999737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:51.221081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa85623-1bef-4fe0-9c93-de3514999737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:51.222657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa85623-1bef-4fe0-9c93-de3514999737 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:51.240948 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1319/2626] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:54.959057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:54.968999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:55.035007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e95a6-db56-4bc6-95a3-cc181ef33aa2 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:55.041218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:55.042203 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.042631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.043996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] PATH_INFO: `/v3/users/b7d59ca4a62f41c782a67084fe3db56c` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.046049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d59ca4a62f41c782a67084fe3db56c)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:55.049067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:55.108666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d59ca4a62f41c782a67084fe3db56c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.109333 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d59ca4a62f41c782a67084fe3db56c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.118116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d59ca4a62f41c782a67084fe3db56c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.125485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d59ca4a62f41c782a67084fe3db56c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.138584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d59ca4a62f41c782a67084fe3db56c'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.144797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1308/2627] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:54 2022] DELETE /identity/v3/users/b7d59ca4a62f41c782a67084fe3db56c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:55.149368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.149882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.149997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.169810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:55.171045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:55.234163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1320/2628] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:55.244456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:55.245988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:55.320950 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77655e-0ce1-47a2-95c8-a353a9ca41e4 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:55.325176 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd99e7-b1b8-4cab-88a4-79f5797a4535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:55.325795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd99e7-b1b8-4cab-88a4-79f5797a4535 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.326073 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd99e7-b1b8-4cab-88a4-79f5797a4535 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.326372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd99e7-b1b8-4cab-88a4-79f5797a4535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.336206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd99e7-b1b8-4cab-88a4-79f5797a4535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:55.337649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd99e7-b1b8-4cab-88a4-79f5797a4535 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:55.347626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1309/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:55.618835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:55.620458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:55.643788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f48d-cbcd-4ef7-b35e-b0825a045b70 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:55.649167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:55.649902 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.650193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.650469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] PATH_INFO: `/v3/projects/9b55ea5b09f24be4a71958f780d7a783` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.651592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b55ea5b09f24be4a71958f780d7a783)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:55.653534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:55.706964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b55ea5b09f24be4a71958f780d7a783', 'request_id': 'req-a3c41a1a-b6ef-42f6-990a-53830da7ef47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.714858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b55ea5b09f24be4a71958f780d7a783', 'request_id': 'req-a3c41a1a-b6ef-42f6-990a-53830da7ef47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.721225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b55ea5b09f24be4a71958f780d7a783', 'request_id': 'req-a3c41a1a-b6ef-42f6-990a-53830da7ef47'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.733661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] The token cache is being invalidate because project 9b55ea5b09f24be4a71958f780d7a783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:55.733892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.734759 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a3c41a1a-b6ef-42f6-990a-53830da7ef47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:55.736547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1321/2630] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:55 2022] DELETE /identity/v3/projects/9b55ea5b09f24be4a71958f780d7a783 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:55.742802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.743182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.743526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.766115 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:55.767406 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:55.791868 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:55.830107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1310/2631] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:55.841688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.842035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.842420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.866287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:55.867505 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:55.875732 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1322/2632] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:55.880269 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:55.890101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd18bc-c844-43a7-9cd4-73a0185483b5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:55.895532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:55.896132 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.896405 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.896674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.897326 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:55.897622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:55.899219 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:55.914450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1311/2633] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:55.918977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:55.930064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0aaec-04bc-43bf-b3da-975db9760803 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:55.935516 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:55.936306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:55.936580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:55.936911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:55.937534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:55.939518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:55.972519 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1323/2634] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:55 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:55.977268 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:55.988018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653fd3c-c9d9-432e-b2e2-bab341df8937 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:56.044656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:56.045263 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:56.045538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:56.045807 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:56.046468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:56.048342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:56.069587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1312/2635] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:55 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:56.073938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:56.084458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999c45-bce8-4442-853b-e604e2e28aa2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:56.090048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:56.090688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:56.090967 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:56.091242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:56.091907 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:56.093523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:56.104871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1324/2636] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:56.109143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:56.120736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4f89b-a32e-4c2c-8b61-e93360d4be60 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:56.125438 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b95c-0479-40a3-87e4-4fbc81c33dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:56.126041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b95c-0479-40a3-87e4-4fbc81c33dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:56.126352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b95c-0479-40a3-87e4-4fbc81c33dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:56.126629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b95c-0479-40a3-87e4-4fbc81c33dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:56.127278 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773b95c-0479-40a3-87e4-4fbc81c33dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:56.128971 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773b95c-0479-40a3-87e4-4fbc81c33dd0 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:56.139377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1313/2637] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:56.143424 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:56.154018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b415f17-7056-4c49-a5a8-dc08609cd3be admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:56.159450 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:56.160106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:56.160382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:56.160646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] PATH_INFO: `/v3/projects/aa236a91639d48aaa32919e45cf1dd40/users/1f7fe9b4254b4ee391cc6557114ee580/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:56.176367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa236a91639d48aaa32919e45cf1dd40, user_id=1f7fe9b4254b4ee391cc6557114ee580, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:56.179080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:56.188282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:56.193922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1325/2638] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:56 2022] PUT /identity/v3/projects/aa236a91639d48aaa32919e45cf1dd40/users/1f7fe9b4254b4ee391cc6557114ee580/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:56.204164 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:56.204452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:56.204726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:56.231399 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] MFA Rules not processed for user `1f7fe9b4254b4ee391cc6557114ee580`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:56.232612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:56.298627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1314/2639] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:56.309422 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:56.311611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:56.372319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cedd6-dd36-46af-805b-21013a19d76c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:56.376774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160c22-bb63-44d6-b790-48a3746d00c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:56.377391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160c22-bb63-44d6-b790-48a3746d00c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:56.378538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160c22-bb63-44d6-b790-48a3746d00c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:56.378835 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160c22-bb63-44d6-b790-48a3746d00c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:56.388449 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160c22-bb63-44d6-b790-48a3746d00c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:56.389905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160c22-bb63-44d6-b790-48a3746d00c0 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:56.399710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1326/2640] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:57.216525 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:57.218185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:57.247199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:57.285863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ae200-9ae4-4997-9085-9acec810cfeb None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:57.290377 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:57.291013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:57.291288 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:57.291559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] PATH_INFO: `/v3/users/eb497c9ae90f4c248c6bc6fd38d4ce38` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:57.292665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb497c9ae90f4c248c6bc6fd38d4ce38)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:57.294627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:57.342535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb497c9ae90f4c248c6bc6fd38d4ce38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.343188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb497c9ae90f4c248c6bc6fd38d4ce38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.351111 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb497c9ae90f4c248c6bc6fd38d4ce38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.358208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb497c9ae90f4c248c6bc6fd38d4ce38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.370998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb497c9ae90f4c248c6bc6fd38d4ce38'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.377074 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1315/2641] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:57 2022] DELETE /identity/v3/users/eb497c9ae90f4c248c6bc6fd38d4ce38 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:57.381740 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:57.382278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:57.382386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:57.401101 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:57.402334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:57.466486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1327/2642] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:57.476519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:57.477918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:57.539079 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27399ba-aef5-427a-b838-b44284f9d34c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:57.543022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:57.543643 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:57.543917 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:57.544186 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:57.553671 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:57.555127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:57.564664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1316/2643] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:57.836033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:57.838138 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:57.858911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8e491-53b9-4d07-a481-33f99e40b84e None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:57.865107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:57.865713 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:57.866056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:57.866380 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] PATH_INFO: `/v3/projects/0ac60e337fda427790fdee195027f174` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:57.867728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac60e337fda427790fdee195027f174)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:57.869627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:57.921413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac60e337fda427790fdee195027f174', 'request_id': 'req-317ba9b5-0b94-4d83-9b73-e78cdada198f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.929206 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac60e337fda427790fdee195027f174', 'request_id': 'req-317ba9b5-0b94-4d83-9b73-e78cdada198f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.935743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac60e337fda427790fdee195027f174', 'request_id': 'req-317ba9b5-0b94-4d83-9b73-e78cdada198f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.946420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] The token cache is being invalidate because project 0ac60e337fda427790fdee195027f174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:57.946711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.947390 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:57.948875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1328/2644] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:57 2022] DELETE /identity/v3/projects/0ac60e337fda427790fdee195027f174 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:57.952966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:57.962596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371681da-8489-4ba6-ba70-1de4db7a86fe service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.012167 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:58.031550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.032518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.032966 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.033319 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] PATH_INFO: `/v3/users/36b57f0d07224be6800445e23d7ccaa4` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.034809 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b57f0d07224be6800445e23d7ccaa4)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.037845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.108199 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b57f0d07224be6800445e23d7ccaa4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.108870 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b57f0d07224be6800445e23d7ccaa4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.117515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b57f0d07224be6800445e23d7ccaa4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.124479 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b57f0d07224be6800445e23d7ccaa4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.136531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3718a4e5-cb17-4768-815c-28e107871128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b57f0d07224be6800445e23d7ccaa4'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.142621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1317/2645] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:47:57 2022] DELETE /identity/v3/users/36b57f0d07224be6800445e23d7ccaa4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:47:58.431861 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:58.433471 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:58.446912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ba9b5-0b94-4d83-9b73-e78cdada198f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.468023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:58.507038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.507684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.507965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.508240 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] PATH_INFO: `/v3/projects/7a5cffa94c5b4c15a38ebcd2909b1679` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.509492 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5cffa94c5b4c15a38ebcd2909b1679)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.511508 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.559786 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5cffa94c5b4c15a38ebcd2909b1679', 'request_id': 'req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.568624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5cffa94c5b4c15a38ebcd2909b1679', 'request_id': 'req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.575174 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5cffa94c5b4c15a38ebcd2909b1679', 'request_id': 'req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.586675 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] The token cache is being invalidate because project 7a5cffa94c5b4c15a38ebcd2909b1679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:47:58.586963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.587672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6f6d6711-2ec9-4b7c-9837-b23d8ded33dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:47:58.589201 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1329/2646] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:47:58 2022] DELETE /identity/v3/projects/7a5cffa94c5b4c15a38ebcd2909b1679 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:58.594518 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.595049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.595150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.615298 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:58.616597 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:58.680147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1318/2647] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:58.688242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.688533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.688810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.708515 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:58.709650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:58.719540 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1330/2648] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:58.724527 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:58.734359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6b89-1df5-4cbc-a01d-291c14e1a8ff None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.739418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.740026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.740300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.740570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.741218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:47:58.741514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.743135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.756515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1319/2649] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:47:58.762514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:58.777324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae92e4-e8ac-48c4-a5cc-50f31bc76c04 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.782818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.783469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.783752 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.784049 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.784677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.786644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.816497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1331/2650] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:58 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:58.820692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:58.830827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe86dc1-e688-460d-a078-dcd02d827788 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.880926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:58.892283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.892928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.893236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.893702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.894372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.896355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.919639 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1320/2651] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:58 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:58.923971 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:58.934642 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cbadc-3d8a-4c73-926f-70817e24aac9 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.940033 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.940709 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.940987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.941261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.941914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.943577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.953787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1332/2652] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:47:58.957836 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:58.968306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808481d-d2a6-4139-b8cc-cfff84e8a99a admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:58.973840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395aa78e-e763-4bc7-99d1-f59e88e6a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:58.974481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aa78e-e763-4bc7-99d1-f59e88e6a898 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:58.974872 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aa78e-e763-4bc7-99d1-f59e88e6a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:58.975005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aa78e-e763-4bc7-99d1-f59e88e6a898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:58.975661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395aa78e-e763-4bc7-99d1-f59e88e6a898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:58.977282 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395aa78e-e763-4bc7-99d1-f59e88e6a898 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:58.988026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1321/2653] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:58.992559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.002841 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e706-5421-4b78-8872-9cccf140a95a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.007898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.008558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.008831 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.009096 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] PATH_INFO: `/v3/projects/ae015fbd2ca14fecbdcc9b6fc6688c2d/users/ebebaa6c05d34ecca0a9cfcde53801a0/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.023929 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae015fbd2ca14fecbdcc9b6fc6688c2d, user_id=ebebaa6c05d34ecca0a9cfcde53801a0, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.026584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.043511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1333/2654] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:58 2022] PUT /identity/v3/projects/ae015fbd2ca14fecbdcc9b6fc6688c2d/users/ebebaa6c05d34ecca0a9cfcde53801a0/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:59.054727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.055018 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.055296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.084976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] MFA Rules not processed for user `ebebaa6c05d34ecca0a9cfcde53801a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:59.086244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:59.155975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1322/2655] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:59.160737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.213068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:59.225882 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23a744-6e16-4687-bd03-43995f3fec28 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.231911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.232528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.232810 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.233085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.233718 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.235779 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.265926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1334/2656] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:47:59 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:59.270385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.282100 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36becb-5712-4350-8a7b-acdc22533948 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.338261 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.338869 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.339143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.339409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.340030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.341968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.364822 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1323/2657] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:47:59 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:47:59.369055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.379724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1752c-b3e4-4dcb-94a0-322d4aac5512 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.384614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.385205 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.385480 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.385755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.386434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.388024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.398448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1335/2658] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:59.402612 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.413213 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206570e-ce40-43f1-9935-fa2dcaf29980 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.418236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.418906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.419184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.419455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] PATH_INFO: `/v3/projects/e051a570bd31487b8357b7345cf6a034/users/52697417f8de4706ba9261814dd05a7f/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.436231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e051a570bd31487b8357b7345cf6a034, user_id=52697417f8de4706ba9261814dd05a7f, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.440417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.458574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1324/2659] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:59 2022] PUT /identity/v3/projects/e051a570bd31487b8357b7345cf6a034/users/52697417f8de4706ba9261814dd05a7f/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:59.462739 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.473382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8d72a-b2d2-4db5-a240-18326b57a6c0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.533064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb5bac-70d9-49a1-949a-adff37309904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.533603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb5bac-70d9-49a1-949a-adff37309904 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.533879 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb5bac-70d9-49a1-949a-adff37309904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.534147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb5bac-70d9-49a1-949a-adff37309904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.534888 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb5bac-70d9-49a1-949a-adff37309904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.536490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb5bac-70d9-49a1-949a-adff37309904 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.547367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1336/2660] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:47:59.553649 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.564317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5849a-2410-40b7-9f04-5a033a72eb8c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.570425 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.571078 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.571358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.571634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] PATH_INFO: `/v3/projects/e051a570bd31487b8357b7345cf6a034/users/52697417f8de4706ba9261814dd05a7f/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.573895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e051a570bd31487b8357b7345cf6a034, user_id=52697417f8de4706ba9261814dd05a7f, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.576626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.587764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1325/2661] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:47:59 2022] PUT /identity/v3/projects/e051a570bd31487b8357b7345cf6a034/users/52697417f8de4706ba9261814dd05a7f/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:47:59.599147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.599414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.599711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.631119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] MFA Rules not processed for user `52697417f8de4706ba9261814dd05a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:47:59.632330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:59.713183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1337/2662] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:47:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:47:59.725085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.727188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:47:59.743661 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:59.790240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9a36-44af-4644-bf2c-6ab8a72ce77e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.796732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd194ee-460e-4b3b-9480-ae8eae4f895e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.797581 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd194ee-460e-4b3b-9480-ae8eae4f895e service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.797910 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd194ee-460e-4b3b-9480-ae8eae4f895e service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.798295 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd194ee-460e-4b3b-9480-ae8eae4f895e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.813201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd194ee-460e-4b3b-9480-ae8eae4f895e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.815322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd194ee-460e-4b3b-9480-ae8eae4f895e service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.825294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1326/2663] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:47:59.907531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:47:59.918579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2a719-9aa2-49e3-9393-a4e38f9a109d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:47:59.922456 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:47:59.923081 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:47:59.923381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:47:59.923650 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:47:59.969562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:47:59.983940 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:47:59.985420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:47:59.994935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1338/2664] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:47:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:05.475280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:05.475844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:05.475844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:05.502894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:05.504727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:05.573258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1327/2665] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 9 02:48:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:05.587746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:05.589511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:05.687182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e90ae-6cfe-41b8-b989-1f929a5620e9 service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:05.692711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:05.693326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:05.693728 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:05.694006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:05.705343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:05.707058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:05.715565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1339/2666] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:48:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:48:12.289010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:12.300095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c871cf-4ea5-4f58-b684-52699013f20d None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:12.303778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:12.304387 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:12.304709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:12.304982 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:12.306860 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:12.381122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:12.382899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94018d-5ce5-45d5-b67e-df7489420548 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:12.393207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1328/2667] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:15.712093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=39 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:17.145370 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:17.204606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdafb2-5f97-4a20-9c40-9ad5b5b24c71 service swift] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:17.209235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:17.209870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:17.210190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:17.210469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] PATH_INFO: `/v3/users/ebebaa6c05d34ecca0a9cfcde53801a0` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:17.211673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebebaa6c05d34ecca0a9cfcde53801a0)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:17.213715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:17.264473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebebaa6c05d34ecca0a9cfcde53801a0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.271923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebebaa6c05d34ecca0a9cfcde53801a0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.272558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebebaa6c05d34ecca0a9cfcde53801a0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.280002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebebaa6c05d34ecca0a9cfcde53801a0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.292261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebebaa6c05d34ecca0a9cfcde53801a0'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.299073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1340/2668] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:17 2022] DELETE /identity/v3/users/ebebaa6c05d34ecca0a9cfcde53801a0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:17.303867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:17.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:17.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:17.324312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:17.325538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:17.383852 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=100 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:17.392881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1329/2669] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:17.402901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:17.404414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:17.465912 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c3e94-3fca-45c3-9c3b-517d07467a33 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:17.469862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:17.470507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:17.470788 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:17.471078 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:17.480556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:17.482010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:17.492572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1341/2670] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:17.775731 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:17.777218 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:17.789412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600147d8-5715-4c73-a8a1-13983b6a573b None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:17.794181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:17.794832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:17.795109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:17.795385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] PATH_INFO: `/v3/projects/ae015fbd2ca14fecbdcc9b6fc6688c2d` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:17.796500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae015fbd2ca14fecbdcc9b6fc6688c2d)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:17.798634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:17.846885 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae015fbd2ca14fecbdcc9b6fc6688c2d', 'request_id': 'req-1b65629b-a1b3-4fb1-9f79-cc977b522856'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.855289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae015fbd2ca14fecbdcc9b6fc6688c2d', 'request_id': 'req-1b65629b-a1b3-4fb1-9f79-cc977b522856'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.863635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae015fbd2ca14fecbdcc9b6fc6688c2d', 'request_id': 'req-1b65629b-a1b3-4fb1-9f79-cc977b522856'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.880307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] The token cache is being invalidate because project ae015fbd2ca14fecbdcc9b6fc6688c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:17.880603 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.881562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:17.883895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1330/2671] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:17 2022] DELETE /identity/v3/projects/ae015fbd2ca14fecbdcc9b6fc6688c2d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:17.888226 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:17.901632 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef682f9f-da39-41d1-adf0-93ecfc0563d4 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:17.920781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:17.963468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:17.964195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:17.964477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:17.964750 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] PATH_INFO: `/v3/users/52697417f8de4706ba9261814dd05a7f` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:17.966300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52697417f8de4706ba9261814dd05a7f)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:17.968119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.020181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52697417f8de4706ba9261814dd05a7f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.029681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52697417f8de4706ba9261814dd05a7f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.030497 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52697417f8de4706ba9261814dd05a7f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.039056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52697417f8de4706ba9261814dd05a7f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.051704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-02fa8909-a083-4e68-835b-94fee3cfa9b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52697417f8de4706ba9261814dd05a7f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.057863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1342/2672] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:17 2022] DELETE /identity/v3/users/52697417f8de4706ba9261814dd05a7f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:18.327522 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.329156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:18.343197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65629b-a1b3-4fb1-9f79-cc977b522856 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.405031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.406047 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.406541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.406988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] PATH_INFO: `/v3/projects/e051a570bd31487b8357b7345cf6a034` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.408871 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e051a570bd31487b8357b7345cf6a034)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.411312 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.459099 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e051a570bd31487b8357b7345cf6a034', 'request_id': 'req-4292dd41-b7e3-4577-b99b-c6df5e59b985'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.465065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e051a570bd31487b8357b7345cf6a034', 'request_id': 'req-4292dd41-b7e3-4577-b99b-c6df5e59b985'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.470367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e051a570bd31487b8357b7345cf6a034', 'request_id': 'req-4292dd41-b7e3-4577-b99b-c6df5e59b985'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.484684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] The token cache is being invalidate because project e051a570bd31487b8357b7345cf6a034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:18.484976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.485613 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-4292dd41-b7e3-4577-b99b-c6df5e59b985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:18.487294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1331/2673] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:18 2022] DELETE /identity/v3/projects/e051a570bd31487b8357b7345cf6a034 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:18.492806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.493329 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.493452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.514219 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:18.515371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:18.585763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1343/2674] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:18.595094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.595514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.595867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.624553 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:18.625763 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:18.635225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1332/2675] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:18.639617 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.650285 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a439-88ef-4560-bf14-2cfe80139087 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.656372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.656982 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.657321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.657589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.658274 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:48:18.658580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.660229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.672606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1344/2676] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:48:18.677222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.687223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466594d1-93b8-485f-95f3-cf9421d6881a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.693062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.693664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.693938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.694234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.694856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.696795 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.701492 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:18.727529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1333/2677] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:48:18 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:18.731914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.742782 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73275a38-941b-4c95-98af-7962716d176d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.770944 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:18.803579 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.804196 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.804477 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.804751 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.805377 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.807421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.829926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1345/2678] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:48:18 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:18.834185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.845428 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e7712-7cf3-4090-9f7c-7384561c00de admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.850512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.851116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.851396 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.851669 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.852317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.853985 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.864960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1334/2679] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:18.869004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.878964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c47fc-dcf6-4d3f-9e8a-e4a551659109 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.884088 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe36dfa-c357-4649-84f4-41c3dbed410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.884676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe36dfa-c357-4649-84f4-41c3dbed410e admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.884918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe36dfa-c357-4649-84f4-41c3dbed410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.885207 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe36dfa-c357-4649-84f4-41c3dbed410e admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.885867 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe36dfa-c357-4649-84f4-41c3dbed410e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.887529 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe36dfa-c357-4649-84f4-41c3dbed410e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.897757 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1346/2680] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:18.902135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:18.913120 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcba2d-6191-4d02-9182-67d3079456ed admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:18.918033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:18.918699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.918975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.919244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] PATH_INFO: `/v3/projects/0b15306590844401b53d3d82bd5bd242/users/af3f4bc22f6d40b9b6b5514753c6b3a3/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:18.933825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b15306590844401b53d3d82bd5bd242, user_id=af3f4bc22f6d40b9b6b5514753c6b3a3, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:18.936516 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:18.955025 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1335/2681] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:18 2022] PUT /identity/v3/projects/0b15306590844401b53d3d82bd5bd242/users/af3f4bc22f6d40b9b6b5514753c6b3a3/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:18.971826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:18.972112 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:18.972383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.003454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] MFA Rules not processed for user `af3f4bc22f6d40b9b6b5514753c6b3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:19.004717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:19.069627 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1347/2682] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:19.073979 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.136173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf013d7-d7d9-4bab-96be-7a894ae5d4a7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.141647 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.142385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.142715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.142996 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.143678 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.145628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.177578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1336/2683] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:48:19 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:19.182615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.197223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c0db5-2810-4691-babd-2e853268fcef None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.257401 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.258025 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.258340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.258616 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.259372 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.261303 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.284307 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1348/2684] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:48:19 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:19.288448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.299126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aac06-1bd7-4faa-afb2-d5ad3c629305 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.303791 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.304389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.304667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.304936 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.305609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.307367 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.317699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1337/2685] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:19.322019 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.332487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1ef1d-013b-4b40-94ae-b32f4e997867 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.338444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.339099 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.339381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.339651 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] PATH_INFO: `/v3/projects/936f363595bb4dc59c609e9f929bd20b/users/fea1e4521fe240e585fe0c9f4c762179/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.355899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=936f363595bb4dc59c609e9f929bd20b, user_id=fea1e4521fe240e585fe0c9f4c762179, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.358611 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.374649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1349/2686] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:19 2022] PUT /identity/v3/projects/936f363595bb4dc59c609e9f929bd20b/users/fea1e4521fe240e585fe0c9f4c762179/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:19.378893 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.389128 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a0d61-a3a3-4f58-8c3d-2d893e122c9b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.453897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02efa1-00f5-45ae-8e14-f86b2e0954f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.454543 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02efa1-00f5-45ae-8e14-f86b2e0954f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.454846 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02efa1-00f5-45ae-8e14-f86b2e0954f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.455121 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02efa1-00f5-45ae-8e14-f86b2e0954f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.455768 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02efa1-00f5-45ae-8e14-f86b2e0954f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.457358 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02efa1-00f5-45ae-8e14-f86b2e0954f6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.467060 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1338/2687] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:19.471969 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.477687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:19.483549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fda378-1ac5-4533-8baa-ed1244701fff admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.490051 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.490805 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.491084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.491354 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] PATH_INFO: `/v3/projects/936f363595bb4dc59c609e9f929bd20b/users/fea1e4521fe240e585fe0c9f4c762179/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.493296 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=936f363595bb4dc59c609e9f929bd20b, user_id=fea1e4521fe240e585fe0c9f4c762179, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.496084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.506304 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1350/2688] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:19 2022] PUT /identity/v3/projects/936f363595bb4dc59c609e9f929bd20b/users/fea1e4521fe240e585fe0c9f4c762179/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:19.516878 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.517174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.517481 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.547414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] MFA Rules not processed for user `fea1e4521fe240e585fe0c9f4c762179`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:19.548675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:19.557988 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:19.620754 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1339/2689] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:19.632663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.634814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:19.694626 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8a679-129a-4bc5-a22c-14e4cd636a4a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.698726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c377c9f-3d3e-4a59-9505-eed3d4790597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.699331 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c377c9f-3d3e-4a59-9505-eed3d4790597 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.699612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c377c9f-3d3e-4a59-9505-eed3d4790597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.699930 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c377c9f-3d3e-4a59-9505-eed3d4790597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.709634 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c377c9f-3d3e-4a59-9505-eed3d4790597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.711126 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c377c9f-3d3e-4a59-9505-eed3d4790597 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.720710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1351/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:19.809180 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:19.819472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4636cf-60ca-4f09-ac67-9c07680b904a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:19.823559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:19.824166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:19.824443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:19.824707 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:19.881586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:19.882991 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:19.892028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1340/2691] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:20.428673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c0661-97f7-4187-b6aa-9af537f4950e None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:20.428964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c0661-97f7-4187-b6aa-9af537f4950e None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:20.429235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c0661-97f7-4187-b6aa-9af537f4950e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:20.448848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-ac0c0661-97f7-4187-b6aa-9af537f4950e None None] MFA Rules not processed for user `90fa94f90d49485a81d502929bcd0b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:20.450726 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-ac0c0661-97f7-4187-b6aa-9af537f4950e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:20.515796 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1352/2692] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:21.258816 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:21.260448 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:21.372314 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771f9c8-be28-479b-8b0e-558b7ff20736 service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:21.378437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:21.379138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:21.379552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:21.379876 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] PATH_INFO: `/v3/users/1f7fe9b4254b4ee391cc6557114ee580` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:21.381106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7fe9b4254b4ee391cc6557114ee580)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:21.383114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:21.446735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:21.447502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7fe9b4254b4ee391cc6557114ee580'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:21.448150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7fe9b4254b4ee391cc6557114ee580'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:21.457402 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7fe9b4254b4ee391cc6557114ee580'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:21.466997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7fe9b4254b4ee391cc6557114ee580'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:21.481427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7fe9b4254b4ee391cc6557114ee580'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:21.491010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1341/2693] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:21 2022] DELETE /identity/v3/users/1f7fe9b4254b4ee391cc6557114ee580 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:21.492987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:21.493214 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:21.493487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:21.528935 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:21.530887 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:21.601383 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1353/2694] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:21.611792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:21.613356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:21.673718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cbd4a-e32c-4646-aca9-6d534ae38519 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:21.677404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:21.678090 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:21.678433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:21.678706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:21.688701 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:21.690191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:21.702826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1342/2695] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:21.962585 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:21.964293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:21.976321 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8352-b3cb-4af9-9dc2-ba1abf668ff8 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:21.982273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:21.983016 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:21.983306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:21.983550 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] PATH_INFO: `/v3/projects/aa236a91639d48aaa32919e45cf1dd40` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:21.984968 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa236a91639d48aaa32919e45cf1dd40)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:21.987300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:22.032965 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:22.045337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa236a91639d48aaa32919e45cf1dd40', 'request_id': 'req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:22.053852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa236a91639d48aaa32919e45cf1dd40', 'request_id': 'req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:22.060133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa236a91639d48aaa32919e45cf1dd40', 'request_id': 'req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:22.069946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] The token cache is being invalidate because project aa236a91639d48aaa32919e45cf1dd40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:22.070263 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:22.070906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:22.072578 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1354/2696] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:21 2022] DELETE /identity/v3/projects/aa236a91639d48aaa32919e45cf1dd40 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:24.830566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:24.832655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:24.840631 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:24.842197 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:24.844780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b37c0d-c5b3-4206-be5b-7711681ecfe0 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:24.904148 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:24.904771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:24.905049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:24.905323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:24.907153 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:24.932562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0c6bb-d2a2-4bb5-b8a9-eee7787a76d8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:24.937292 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:24.937943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:24.938272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:24.938535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] PATH_INFO: `/v3/users/af3f4bc22f6d40b9b6b5514753c6b3a3` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:24.939677 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3f4bc22f6d40b9b6b5514753c6b3a3)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:24.941635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:24.980566 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:24.982182 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608643f-aab3-4c48-ae14-1038714bc270 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:24.992331 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1343/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:24.996864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3f4bc22f6d40b9b6b5514753c6b3a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.005699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3f4bc22f6d40b9b6b5514753c6b3a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.006408 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3f4bc22f6d40b9b6b5514753c6b3a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.014381 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3f4bc22f6d40b9b6b5514753c6b3a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.025917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3f4bc22f6d40b9b6b5514753c6b3a3'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.032255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1355/2698] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:24 2022] DELETE /identity/v3/users/af3f4bc22f6d40b9b6b5514753c6b3a3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:48:25.036858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:25.037149 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:25.037420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:25.057191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:25.058508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:25.072119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:25.123232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1344/2699] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:25.133031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:25.134565 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:25.180538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:25.197557 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96c41-0cb2-4a2c-b856-b66edfae6473 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:25.201847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:25.202614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:25.202876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:25.203132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:25.213171 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:25.214910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:25.224952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1356/2700] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:25.517674 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:25.519225 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:25.532077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e55e9-93e4-48fd-a75c-b8cc1ff83462 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:25.538296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:25.539207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:25.539720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:25.545775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] PATH_INFO: `/v3/projects/0b15306590844401b53d3d82bd5bd242` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:25.549227 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b15306590844401b53d3d82bd5bd242)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:25.551221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:25.603244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b15306590844401b53d3d82bd5bd242', 'request_id': 'req-356c6aaf-ae92-4379-8d71-c02c26be42f6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.610463 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b15306590844401b53d3d82bd5bd242', 'request_id': 'req-356c6aaf-ae92-4379-8d71-c02c26be42f6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.618684 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b15306590844401b53d3d82bd5bd242', 'request_id': 'req-356c6aaf-ae92-4379-8d71-c02c26be42f6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.633548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] The token cache is being invalidate because project 0b15306590844401b53d3d82bd5bd242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:25.633847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.634621 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.636382 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1345/2701] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:25 2022] DELETE /identity/v3/projects/0b15306590844401b53d3d82bd5bd242 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:25.640571 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:25.650922 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca0b4-65e9-4ecf-b54e-15136d2c8586 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:25.718378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:25.719026 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:25.719306 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:25.719577 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] PATH_INFO: `/v3/users/fea1e4521fe240e585fe0c9f4c762179` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:25.720714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea1e4521fe240e585fe0c9f4c762179)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:25.722672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:25.773592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea1e4521fe240e585fe0c9f4c762179'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.782068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea1e4521fe240e585fe0c9f4c762179'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.782793 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea1e4521fe240e585fe0c9f4c762179'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.789781 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea1e4521fe240e585fe0c9f4c762179'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.802118 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0ace620d-fbd0-431a-a07f-690e8fca7f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea1e4521fe240e585fe0c9f4c762179'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:25.808479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1357/2702] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:25 2022] DELETE /identity/v3/users/fea1e4521fe240e585fe0c9f4c762179 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:26.238076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.246457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:26.268495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c6aaf-ae92-4379-8d71-c02c26be42f6 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.348417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.349389 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.349524 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.349879 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] PATH_INFO: `/v3/projects/936f363595bb4dc59c609e9f929bd20b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.351094 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=936f363595bb4dc59c609e9f929bd20b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.353114 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.410453 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:26.422803 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936f363595bb4dc59c609e9f929bd20b', 'request_id': 'req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:26.432880 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936f363595bb4dc59c609e9f929bd20b', 'request_id': 'req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:26.441338 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936f363595bb4dc59c609e9f929bd20b', 'request_id': 'req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:26.457151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] The token cache is being invalidate because project 936f363595bb4dc59c609e9f929bd20b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:26.457459 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:26.458251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0bb4fd7c-e0fc-4525-beef-f76b96d4d8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:26.460430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1346/2703] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:26 2022] DELETE /identity/v3/projects/936f363595bb4dc59c609e9f929bd20b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:26.465318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.466022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.466022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.491031 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:26.492287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:26.561326 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1358/2704] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:48:26.569359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.569580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.569864 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.595322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:26.596567 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:26.608455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1347/2705] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:26.612993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.618349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:26.624776 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64577c9f-6a69-461d-b849-bd93ae9b165a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.630863 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.631534 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.631905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.632347 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.633013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:48:26.633373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.635109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.647035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1359/2706] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:48:26.651541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.660797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cd8eb-140c-4a26-9fe3-8df03f80cbfd None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.665573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.666205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.666490 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.666756 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.667447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.669335 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.698628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1348/2707] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:48:26 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:26.703815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.714852 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c571a9-3e94-41de-9b2f-1df7bf276625 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.775607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.776232 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.776500 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.776771 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.777440 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.779457 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.804685 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1360/2708] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:48:26 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:26.809177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.820638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440af6-33c9-42e9-bb29-3c2912957956 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.825689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.826412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.826697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.826957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.827605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.829194 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.843281 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1349/2709] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:26.844790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.854990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be04a4-e8a6-4f60-8443-5e41de3174aa admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.859943 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef55749-f4fc-4692-a7ec-08e500f3e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.860551 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef55749-f4fc-4692-a7ec-08e500f3e78a admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.860832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef55749-f4fc-4692-a7ec-08e500f3e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.861106 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef55749-f4fc-4692-a7ec-08e500f3e78a admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.861763 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef55749-f4fc-4692-a7ec-08e500f3e78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.863411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef55749-f4fc-4692-a7ec-08e500f3e78a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.874597 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1361/2710] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:48:26.878907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:26.889371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25133a-1994-4d46-acd2-4feaabfae529 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:26.894293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:26.894958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.895238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.895512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] PATH_INFO: `/v3/projects/d3604d08826944eea8e6236f8a165467/users/37c286a331b04792a4d3a68b435e94b4/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.910412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3604d08826944eea8e6236f8a165467, user_id=37c286a331b04792a4d3a68b435e94b4, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:26.913040 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:26.926964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1350/2711] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:26 2022] PUT /identity/v3/projects/d3604d08826944eea8e6236f8a165467/users/37c286a331b04792a4d3a68b435e94b4/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:26.938413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:26.938699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:26.938970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:26.969084 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] MFA Rules not processed for user `37c286a331b04792a4d3a68b435e94b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:26.970345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:27.036120 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1362/2712] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:27.047747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:27.049670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:27.110947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d3fa2-a652-4bc4-bb10-0fab80ad1cde admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:27.115088 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:27.115714 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:27.116675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:27.116675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:27.125538 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:27.127017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:27.147907 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1351/2713] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:32.331619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:32.333073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:32.405714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:32.432162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94624040-c932-49c2-821a-b7e3810278d0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:32.436162 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:32.436773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:32.437055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:32.437325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:32.439181 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:32.550210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:32.551608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:32.571766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1363/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:35.960465 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:35.962023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:36.061053 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:36.122007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085e105-1d89-4f77-8daf-b70136eb7e26 service glance] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:36.127577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:36.128848 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:36.129134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:36.129493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:36.134050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:36.146001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:36.147622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:36.157770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1352/2715] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:38.739762 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:38.741720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:38.753356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be25692-4a0a-46f9-8610-116f123a59d2 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:38.757105 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:38.757774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:38.758053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:38.758378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:38.760298 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:38.827528 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:38.829006 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:38.839314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1364/2716] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:39.530420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:39.532435 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:39.546761 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984eee2-41e5-4952-b663-16ca7ca58595 service placement] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:39.551545 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:39.552187 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:39.552470 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:39.552746 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] PATH_INFO: `/v3/users/f96475caac28419eb0b7dd78375e9e52` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:39.553874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96475caac28419eb0b7dd78375e9e52)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:39.555841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:39.610559 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96475caac28419eb0b7dd78375e9e52'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.611168 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96475caac28419eb0b7dd78375e9e52'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.620511 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96475caac28419eb0b7dd78375e9e52'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.628293 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96475caac28419eb0b7dd78375e9e52'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.641384 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96475caac28419eb0b7dd78375e9e52'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.652366 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1353/2717] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:39 2022] DELETE /identity/v3/users/f96475caac28419eb0b7dd78375e9e52 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:48:39.840429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:39.857563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ff5c-7f7e-4fa7-80a3-b3b15928819b service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:39.880963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:39.921513 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:39.922488 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:39.922923 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:39.923356 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] PATH_INFO: `/v3/projects/2c5d24d2b8dc4844a48a76ed2ea2cffe` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:39.925053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5d24d2b8dc4844a48a76ed2ea2cffe)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:39.927167 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:39.978749 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5d24d2b8dc4844a48a76ed2ea2cffe', 'request_id': 'req-e5b841e0-803a-4df8-bb96-f527fd028fe1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.989648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5d24d2b8dc4844a48a76ed2ea2cffe', 'request_id': 'req-e5b841e0-803a-4df8-bb96-f527fd028fe1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:39.996369 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5d24d2b8dc4844a48a76ed2ea2cffe', 'request_id': 'req-e5b841e0-803a-4df8-bb96-f527fd028fe1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.008523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] The token cache is being invalidate because project 2c5d24d2b8dc4844a48a76ed2ea2cffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:40.008817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.009664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.011297 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1365/2718] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:39 2022] DELETE /identity/v3/projects/2c5d24d2b8dc4844a48a76ed2ea2cffe => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:40.015372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:40.016857 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:40.031797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2bd21-bbf0-4ebc-b982-2e050b07a3ad admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:40.090427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:40.091173 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:40.091452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:40.091723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] PATH_INFO: `/v3/users/cd87cc44a7a84e36b26596504f2d1cb6` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:40.092963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd87cc44a7a84e36b26596504f2d1cb6)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:40.094921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:40.133845 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:40.149423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd87cc44a7a84e36b26596504f2d1cb6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.150123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd87cc44a7a84e36b26596504f2d1cb6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.158130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd87cc44a7a84e36b26596504f2d1cb6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.165062 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd87cc44a7a84e36b26596504f2d1cb6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.176904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd87cc44a7a84e36b26596504f2d1cb6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.182990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1354/2719] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:40 2022] DELETE /identity/v3/users/cd87cc44a7a84e36b26596504f2d1cb6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:40.451666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:40.453287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:40.467890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b841e0-803a-4df8-bb96-f527fd028fe1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:40.564576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:40.565272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:40.565531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:40.565807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] PATH_INFO: `/v3/projects/d2e79b76b02a448f921991e7a0188871` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:40.567172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e79b76b02a448f921991e7a0188871)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:40.569314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:40.655344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e79b76b02a448f921991e7a0188871', 'request_id': 'req-67890436-2fc9-4ccf-ad97-f039a95942a8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.664382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e79b76b02a448f921991e7a0188871', 'request_id': 'req-67890436-2fc9-4ccf-ad97-f039a95942a8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.667994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:40.678056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e79b76b02a448f921991e7a0188871', 'request_id': 'req-67890436-2fc9-4ccf-ad97-f039a95942a8'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.693934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] The token cache is being invalidate because project d2e79b76b02a448f921991e7a0188871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:40.694262 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.694926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:40.696453 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1366/2720] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:40 2022] DELETE /identity/v3/projects/d2e79b76b02a448f921991e7a0188871 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:41.439059 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:41.441004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:41.521520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91704f-0bca-46f5-b2b4-c5cb501befa7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:41.584211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:41.585371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:41.585371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:41.585371 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:41.587028 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:41.600627 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:41.602309 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f7ac-b799-4f50-9697-94f189bba14b service placement] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:41.613606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1355/2721] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:45.936158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:45.938067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:46.011798 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67890436-2fc9-4ccf-ad97-f039a95942a8 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:46.016573 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:46.017228 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.017511 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.017789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] PATH_INFO: `/v3/users/37c286a331b04792a4d3a68b435e94b4` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.018985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=37c286a331b04792a4d3a68b435e94b4)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:46.020964 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:46.104111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37c286a331b04792a4d3a68b435e94b4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.116556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37c286a331b04792a4d3a68b435e94b4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.117346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37c286a331b04792a4d3a68b435e94b4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.127415 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37c286a331b04792a4d3a68b435e94b4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.143536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37c286a331b04792a4d3a68b435e94b4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.151827 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1367/2722] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:45 2022] DELETE /identity/v3/users/37c286a331b04792a4d3a68b435e94b4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:46.156380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.156896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.157009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.197502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:46.198914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:46.241783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:46.278641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1356/2723] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:46.288535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:46.290022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:46.347727 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb1e4-f215-499e-9903-4d7092f2a0dd admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:46.351588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125dd30-9b89-4aff-b46f-bf51344d25ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:46.352265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125dd30-9b89-4aff-b46f-bf51344d25ad service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.352546 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125dd30-9b89-4aff-b46f-bf51344d25ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.352938 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125dd30-9b89-4aff-b46f-bf51344d25ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.362724 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125dd30-9b89-4aff-b46f-bf51344d25ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:46.364673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125dd30-9b89-4aff-b46f-bf51344d25ad service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:46.374892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1368/2724] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:46.668837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:46.670562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:46.684894 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b6250-0e50-478a-a00e-3f433bb5c3b9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:46.690586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:46.691238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.691510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.691775 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] PATH_INFO: `/v3/projects/d3604d08826944eea8e6236f8a165467` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.693020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3604d08826944eea8e6236f8a165467)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:46.694964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:46.748019 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3604d08826944eea8e6236f8a165467', 'request_id': 'req-deb3ff40-3ebd-4f05-897f-79f351f71d99'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.756002 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3604d08826944eea8e6236f8a165467', 'request_id': 'req-deb3ff40-3ebd-4f05-897f-79f351f71d99'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.762251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3604d08826944eea8e6236f8a165467', 'request_id': 'req-deb3ff40-3ebd-4f05-897f-79f351f71d99'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.776931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] The token cache is being invalidate because project d3604d08826944eea8e6236f8a165467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:46.777266 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.777930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-deb3ff40-3ebd-4f05-897f-79f351f71d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:46.779486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1357/2725] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:46 2022] DELETE /identity/v3/projects/d3604d08826944eea8e6236f8a165467 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:46.784775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.785140 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.785509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.798191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:46.842704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:46.843915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:46.914208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1369/2726] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:48:46.922322 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.922954 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.923452 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.947778 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:46.949024 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:46.957850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1358/2727] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:46.962294 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:46.974406 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ed29-82d4-4b63-ac6b-d58920444280 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:46.979233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:46.979973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:46.980355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:46.980722 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:46.981592 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:48:46.981924 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:46.983873 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:46.998179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1370/2728] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:48:47.002469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:47.015642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c5154-5a98-425a-9a51-8a4cbd00ec52 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:47.017829 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:47.018554 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.018831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.019146 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.019920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:47.021942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:47.059688 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1359/2729] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:48:47 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:47.064402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:47.077491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799da88-0b3c-457b-b37d-caa8dd71b6ad admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:47.144319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:47.145145 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.145429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.145701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.146375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:47.148314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:47.178217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1371/2730] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:48:47 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:48:47.180927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:47.192615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905efa02-7c9c-4f79-9034-52a2ba613c4b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:47.197277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:47.197889 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.198185 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.198503 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.199471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:47.201077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:47.211990 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1360/2731] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:47.216175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:47.226895 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4bc8-51ad-42ef-a90e-1ba36aa66e59 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:47.232332 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392da04-bb6b-4276-a293-76c30bb83b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:47.233023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392da04-bb6b-4276-a293-76c30bb83b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.233397 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392da04-bb6b-4276-a293-76c30bb83b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.233672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392da04-bb6b-4276-a293-76c30bb83b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.234374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392da04-bb6b-4276-a293-76c30bb83b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:47.235987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392da04-bb6b-4276-a293-76c30bb83b4d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:47.247465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1372/2732] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:47.251715 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:47.262104 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf96ce8-3054-43b1-aca5-a3321c713741 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:47.266811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:47.267467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.267744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.268011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] PATH_INFO: `/v3/projects/9fbabfa2e2734fc099a648662ec9472d/users/1192cc9ef0e840c187897a9359ebee14/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.282859 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fbabfa2e2734fc099a648662ec9472d, user_id=1192cc9ef0e840c187897a9359ebee14, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:47.285482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:47.299161 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1361/2733] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:47 2022] PUT /identity/v3/projects/9fbabfa2e2734fc099a648662ec9472d/users/1192cc9ef0e840c187897a9359ebee14/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:47.368389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.368672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.368945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.396584 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] MFA Rules not processed for user `1192cc9ef0e840c187897a9359ebee14`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:47.397845 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:47.447039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:47.475952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1373/2734] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:47.487739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:47.489856 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:47.551699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a4130-c7cf-4a86-b99c-6fec0cb50699 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:47.555451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a0b055ed0d14d3e9428d1291f0f1209', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:47.556067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:47.556341 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:47.556606 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:47.566007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:47.567454 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:47.571087 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:47.577280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1362/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:49.952854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:49.954355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:50.035799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30400e5b-014b-4a33-939d-8bb04773fe41 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:50.040509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:50.041250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:50.041671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:50.041945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:50.044108 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:50.120243 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:50.121689 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:50.135060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1374/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:53.061363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.065720 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:53.087502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b29a9-a6fb-4c05-be79-0f8f712eab2f service glance] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.094365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.095297 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.096205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.096592 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] PATH_INFO: `/v3/users/81447e83a97b4519ac6d0eac9a06bdef` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.098023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81447e83a97b4519ac6d0eac9a06bdef)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.100744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.163887 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81447e83a97b4519ac6d0eac9a06bdef'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.164500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81447e83a97b4519ac6d0eac9a06bdef'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.172710 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81447e83a97b4519ac6d0eac9a06bdef'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.179681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81447e83a97b4519ac6d0eac9a06bdef'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.191023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-3478ca3c-f814-48cc-b5b2-86a17aa2119c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81447e83a97b4519ac6d0eac9a06bdef'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.201722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1363/2737] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:48:53 2022] DELETE /identity/v3/users/81447e83a97b4519ac6d0eac9a06bdef => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:53.392282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.403317 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c260c-097d-4fd9-b19e-9b43823851df service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.473666 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.474359 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.474641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.474915 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] PATH_INFO: `/v3/projects/ca6695b7eb3b40ecb4d46292a95e6122` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.476068 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6695b7eb3b40ecb4d46292a95e6122)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.477997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.489208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:53.533233 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6695b7eb3b40ecb4d46292a95e6122', 'request_id': 'req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.543153 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6695b7eb3b40ecb4d46292a95e6122', 'request_id': 'req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.552048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6695b7eb3b40ecb4d46292a95e6122', 'request_id': 'req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.564465 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] The token cache is being invalidate because project ca6695b7eb3b40ecb4d46292a95e6122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:48:53.564755 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.565416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-63aebf4c-0b73-4318-82f1-6ca3b0791e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:48:53.567008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1375/2738] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:48:53 2022] DELETE /identity/v3/projects/ca6695b7eb3b40ecb4d46292a95e6122 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:53.573469 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.573759 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.574031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.593989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:53.595223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:53.663526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1364/2739] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:53.671927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.672209 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.672474 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.695129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:53.697022 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:53.708421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1376/2740] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:48:53.712708 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.722951 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a1200-ddb6-4961-9b77-ec8d02f3bde3 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.728740 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.729347 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.729654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.729929 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.730620 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:48:53.730922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.732515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.745364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1365/2741] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:48:53.751997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.766239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d16e-a6e5-4947-81d7-d0baf59f9cf0 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.772754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.773349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.773628 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.773897 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.774566 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.776454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.805344 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1377/2742] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:48:53 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:53.809508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.819580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410b88-a855-469b-9f5e-124e474cbe17 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.875497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:53.883449 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.884122 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.884672 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.885154 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.885849 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.888623 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.912200 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1366/2743] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:48:53 2022] POST /identity/v3/users => generated 279 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:53.916322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.926721 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c940f-f7a3-498e-ba70-4f4a07896729 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.931816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.932427 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.932706 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.932977 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.933625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.935248 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.945970 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1378/2744] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:53.950077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.960398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1cacc-433b-4d77-93ce-3e96e5af4402 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.965299 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98757695-2950-44f4-ae64-c24bffc68988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:53.965895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98757695-2950-44f4-ae64-c24bffc68988 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:53.966177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98757695-2950-44f4-ae64-c24bffc68988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:53.966462 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98757695-2950-44f4-ae64-c24bffc68988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:53.967109 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98757695-2950-44f4-ae64-c24bffc68988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:53.968694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98757695-2950-44f4-ae64-c24bffc68988 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:53.979239 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1367/2745] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:53.983452 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:53.994102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe4a7-1de7-4a7c-8a8f-952552389eba admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:53.999806 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.000490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.000774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.001046 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] PATH_INFO: `/v3/projects/e4fe8ba472be45e2b2d328d8f85cfbee/users/980a49cfef524a6e90bbbce8a959db66/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.016366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fe8ba472be45e2b2d328d8f85cfbee, user_id=980a49cfef524a6e90bbbce8a959db66, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.019044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.034417 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1379/2746] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:53 2022] PUT /identity/v3/projects/e4fe8ba472be45e2b2d328d8f85cfbee/users/980a49cfef524a6e90bbbce8a959db66/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:48:54.047656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.048029 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.048436 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.083245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] MFA Rules not processed for user `980a49cfef524a6e90bbbce8a959db66`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:54.084431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:54.156001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1368/2747] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:54.160849 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.227157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf5e3d-5f75-4839-a5e5-a711f696d4be admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.232981 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.233591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.233864 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.234132 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.234807 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.236708 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.269178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1380/2748] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:48:54 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:48:54.273346 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.284504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62aa76-237d-418c-b8b5-4398574b7c1f None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.343796 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72130308-a188-4d87-a145-89597e11710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.344418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72130308-a188-4d87-a145-89597e11710c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.344693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72130308-a188-4d87-a145-89597e11710c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.344960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72130308-a188-4d87-a145-89597e11710c admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.345587 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72130308-a188-4d87-a145-89597e11710c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.347517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72130308-a188-4d87-a145-89597e11710c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.369997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1369/2749] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:48:54 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:48:54.374163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.385625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcf750-a3b7-4e35-890c-b382f1bef9f2 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.391486 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.392316 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.392886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.392997 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.393868 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.396001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.412710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1381/2750] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:54.417292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72130308-a188-4d87-a145-89597e11710c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.427932 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72130308-a188-4d87-a145-89597e11710c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.432967 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.433615 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.433892 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.434201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] PATH_INFO: `/v3/projects/db84abdc35c34aa08aad1ab92c532f1d/users/a843289cabdd42a3bf0f0980f832626e/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.450127 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db84abdc35c34aa08aad1ab92c532f1d, user_id=a843289cabdd42a3bf0f0980f832626e, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.452757 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.467528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1370/2751] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:54 2022] PUT /identity/v3/projects/db84abdc35c34aa08aad1ab92c532f1d/users/a843289cabdd42a3bf0f0980f832626e/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:54.471580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.482859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e0d9-5a0c-466d-b0a0-f793f971fadc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.488090 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:54.544142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d545f-a9b6-4317-b1a2-6384d2686b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.544777 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d545f-a9b6-4317-b1a2-6384d2686b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.545074 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d545f-a9b6-4317-b1a2-6384d2686b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.545338 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d545f-a9b6-4317-b1a2-6384d2686b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.545986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d545f-a9b6-4317-b1a2-6384d2686b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.547620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d545f-a9b6-4317-b1a2-6384d2686b8f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.557531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1382/2752] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:48:54.561882 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.572433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade303-d588-48a6-b62f-428fc7a65225 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.577365 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.578045 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.578378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.578653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] PATH_INFO: `/v3/projects/db84abdc35c34aa08aad1ab92c532f1d/users/a843289cabdd42a3bf0f0980f832626e/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.580640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db84abdc35c34aa08aad1ab92c532f1d, user_id=a843289cabdd42a3bf0f0980f832626e, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.583513 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.594404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1371/2753] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:48:54 2022] PUT /identity/v3/projects/db84abdc35c34aa08aad1ab92c532f1d/users/a843289cabdd42a3bf0f0980f832626e/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:48:54.604669 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.604958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.605269 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.633983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] MFA Rules not processed for user `a843289cabdd42a3bf0f0980f832626e`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:48:54.635562 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:54.711454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1383/2754] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:48:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:48:54.723238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.725359 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:48:54.749364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:48:54.806395 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04479e78-2eff-4519-92bd-88b115ebd135 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.810329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.810930 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.811205 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.811475 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.820987 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.822447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:54.832423 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1372/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:48:54.923441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:48:54.934312 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cdfb-33d0-40fa-8983-638b46f67df2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:48:54.938180 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd22bf-de8c-4502-a879-2887583fc015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:48:54.938813 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd22bf-de8c-4502-a879-2887583fc015 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:48:54.939092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd22bf-de8c-4502-a879-2887583fc015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:48:54.939360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd22bf-de8c-4502-a879-2887583fc015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:48:54.996558 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd22bf-de8c-4502-a879-2887583fc015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:48:54.998005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd22bf-de8c-4502-a879-2887583fc015 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:48:55.007018 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1384/2756] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:48:54 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:04.827686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=27 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:05.002891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=87 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:12.209645 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:12.273751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370db59-fc74-4fb3-8d6f-e29e106a2eac service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:12.279494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:12.280192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:12.280555 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:12.280980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] PATH_INFO: `/v3/users/980a49cfef524a6e90bbbce8a959db66` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:12.282447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=980a49cfef524a6e90bbbce8a959db66)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:12.285234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:12.343831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980a49cfef524a6e90bbbce8a959db66'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.344579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980a49cfef524a6e90bbbce8a959db66'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.352280 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980a49cfef524a6e90bbbce8a959db66'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.359417 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980a49cfef524a6e90bbbce8a959db66'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.373578 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980a49cfef524a6e90bbbce8a959db66'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.379942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1373/2757] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:12 2022] DELETE /identity/v3/users/980a49cfef524a6e90bbbce8a959db66 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:12.384753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:12.385260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:12.385364 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:12.404644 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:12.405854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:12.470250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1385/2758] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:12.480709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:12.482259 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:12.549965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633d70-9d9e-48cb-85c8-ceb46e39b0dd admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:12.555155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:12.555815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:12.556091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:12.556363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:12.567521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:12.568969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:12.578837 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1374/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:12.865169 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:12.866695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:12.879290 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72dc2a-0c64-48c1-8a75-8d1e5a3dd745 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:12.884013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:12.884747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:12.885094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:12.885433 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] PATH_INFO: `/v3/projects/e4fe8ba472be45e2b2d328d8f85cfbee` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:12.886646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fe8ba472be45e2b2d328d8f85cfbee)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:12.888619 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:12.937435 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fe8ba472be45e2b2d328d8f85cfbee', 'request_id': 'req-f800e05d-417f-4105-a09a-5b84d500871b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.944990 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fe8ba472be45e2b2d328d8f85cfbee', 'request_id': 'req-f800e05d-417f-4105-a09a-5b84d500871b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.951366 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fe8ba472be45e2b2d328d8f85cfbee', 'request_id': 'req-f800e05d-417f-4105-a09a-5b84d500871b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.961956 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] The token cache is being invalidate because project e4fe8ba472be45e2b2d328d8f85cfbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:12.962293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.962955 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:12.964403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1386/2760] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:12 2022] DELETE /identity/v3/projects/e4fe8ba472be45e2b2d328d8f85cfbee => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:12.968747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:12.978628 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f9377-72b8-4bd7-b689-97eb1d0e4e12 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:12.996881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:13.035662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.036292 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.036573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.036851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] PATH_INFO: `/v3/users/a843289cabdd42a3bf0f0980f832626e` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.037999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a843289cabdd42a3bf0f0980f832626e)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.039958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.088325 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a843289cabdd42a3bf0f0980f832626e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.088980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a843289cabdd42a3bf0f0980f832626e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.097427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a843289cabdd42a3bf0f0980f832626e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.104473 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a843289cabdd42a3bf0f0980f832626e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.118482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-213faf62-74d8-4258-bb3c-b7631fa95556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a843289cabdd42a3bf0f0980f832626e'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.124906 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1375/2761] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:12 2022] DELETE /identity/v3/users/a843289cabdd42a3bf0f0980f832626e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:13.412648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:13.414293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:13.430926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800e05d-417f-4105-a09a-5b84d500871b admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:13.492690 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.493334 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.493639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.493910 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] PATH_INFO: `/v3/projects/db84abdc35c34aa08aad1ab92c532f1d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.495107 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db84abdc35c34aa08aad1ab92c532f1d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.497029 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.529482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:13.546524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db84abdc35c34aa08aad1ab92c532f1d', 'request_id': 'req-6136b70c-3140-46b9-b773-1b8fb5a07ba4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.559520 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db84abdc35c34aa08aad1ab92c532f1d', 'request_id': 'req-6136b70c-3140-46b9-b773-1b8fb5a07ba4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.567339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db84abdc35c34aa08aad1ab92c532f1d', 'request_id': 'req-6136b70c-3140-46b9-b773-1b8fb5a07ba4'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.578911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] The token cache is being invalidate because project db84abdc35c34aa08aad1ab92c532f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:13.579198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.579954 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-6136b70c-3140-46b9-b773-1b8fb5a07ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:13.581618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1387/2762] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:13 2022] DELETE /identity/v3/projects/db84abdc35c34aa08aad1ab92c532f1d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:13.586790 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.587283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.587283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.607784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:13.608986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:13.689703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1376/2763] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:13.700476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.700787 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.701054 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.720823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:13.722050 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:13.730318 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1388/2764] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:13.734638 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:13.745220 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983368b-448d-4e7b-9b80-f76549367b82 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:13.751143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.751753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.752034 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.752313 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.752968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:49:13.753275 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.754927 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.766968 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1377/2765] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:49:13.771166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:13.780454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bab2-d517-489c-8c03-096a03890d7d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:13.786200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.786799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.787069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.787333 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.787960 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.789901 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.820131 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1389/2766] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:13 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:13.826363 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:13.837732 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46b470-69c9-4e70-8795-5b96e1e9c94b admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:13.867183 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:13.897974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.898610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.898897 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.899226 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.899847 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.901762 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.923209 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1378/2767] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:13 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:13.927559 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:13.938343 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885150b-b98d-409e-b6ad-b5ee742e437e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:13.943921 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.944462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.944737 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.945010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.945657 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.948272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.959004 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1390/2768] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:13.963480 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:13.978197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992fe76-6f76-4139-9b82-6e166669431c admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:13.983457 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cecd0-54d8-470d-8da3-42a2cbc9bc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:13.984055 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cecd0-54d8-470d-8da3-42a2cbc9bc25 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:13.984334 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cecd0-54d8-470d-8da3-42a2cbc9bc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:13.984596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cecd0-54d8-470d-8da3-42a2cbc9bc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:13.985267 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cecd0-54d8-470d-8da3-42a2cbc9bc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:13.986921 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cecd0-54d8-470d-8da3-42a2cbc9bc25 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:13.997802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1379/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:14.002010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.012402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5023d3-5fbf-4ba8-bfa1-f74cb8d7cbca admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.017490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.018137 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.018454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.018731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] PATH_INFO: `/v3/projects/b57099246b2242cd833e6fdb505079b6/users/fc0fac5eee834b30a5703d3cfeb96732/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.033358 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57099246b2242cd833e6fdb505079b6, user_id=fc0fac5eee834b30a5703d3cfeb96732, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.036048 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.051065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1391/2770] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:14 2022] PUT /identity/v3/projects/b57099246b2242cd833e6fdb505079b6/users/fc0fac5eee834b30a5703d3cfeb96732/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:14.063821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.064152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.064497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.095315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] MFA Rules not processed for user `fc0fac5eee834b30a5703d3cfeb96732`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:14.096532 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:14.160802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1380/2771] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:14.166833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.231426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372007b-3e30-4ed1-a79d-8af78a6c5a4e admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.236463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.237073 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.237350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.237993 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.238646 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.240537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.275341 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1392/2772] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:14 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:14.280915 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.292302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8551ac-18a2-4f02-b88a-08f8f1d2e67e None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.351697 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.352416 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.352699 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.352975 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.353602 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.355574 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.379204 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1381/2773] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:14 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:49:14.383095 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.395208 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710cef2-db79-4c32-a316-e4dafaf1c4fa admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.400039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.400688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.400966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.401235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.401886 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.403518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.419103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1393/2774] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:14.420412 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.431784 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082d6a4-b704-4b96-ba58-e37b9fb4c17d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.437057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.437723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.438012 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.438348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] PATH_INFO: `/v3/projects/260c9333214441eea6f39c69aef4b071/users/69812b6193e64c3581d5da6d6f1ddf2c/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.452820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260c9333214441eea6f39c69aef4b071, user_id=69812b6193e64c3581d5da6d6f1ddf2c, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.455550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.471890 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1382/2775] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:14 2022] PUT /identity/v3/projects/260c9333214441eea6f39c69aef4b071/users/69812b6193e64c3581d5da6d6f1ddf2c/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:14.475986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.486664 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cc74c-92ef-41f1-ba47-7517af09f46a admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.507911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:14.548172 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86279be-a44b-47b1-9cc7-3e9cf95f097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.548791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86279be-a44b-47b1-9cc7-3e9cf95f097b admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.549070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86279be-a44b-47b1-9cc7-3e9cf95f097b admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.549337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86279be-a44b-47b1-9cc7-3e9cf95f097b admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.549987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86279be-a44b-47b1-9cc7-3e9cf95f097b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.551633 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86279be-a44b-47b1-9cc7-3e9cf95f097b admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.562398 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1394/2776] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:49:14.566905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.572641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:14.579206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ccd7e-63dc-4a0c-892d-77f04cec59c9 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.584677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.585330 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.585650 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.585925 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] PATH_INFO: `/v3/projects/260c9333214441eea6f39c69aef4b071/users/69812b6193e64c3581d5da6d6f1ddf2c/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.588023 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260c9333214441eea6f39c69aef4b071, user_id=69812b6193e64c3581d5da6d6f1ddf2c, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.590703 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.601657 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1383/2777] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:14 2022] PUT /identity/v3/projects/260c9333214441eea6f39c69aef4b071/users/69812b6193e64c3581d5da6d6f1ddf2c/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:14.612461 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.613109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.613385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.641293 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] MFA Rules not processed for user `69812b6193e64c3581d5da6d6f1ddf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:14.642547 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:14.713775 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1395/2778] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:14.724963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.727101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:14.798241 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902fbce-c5a9-4331-a380-0188a18f66c2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.802372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.802976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.803253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.803517 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:14.812908 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:14.814349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:14.823718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1384/2779] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:14.928987 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:14.944514 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8dd1-6023-44d9-a2b4-3558f9197687 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:14.949512 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7d37a-5135-4b23-82a6-0ef708565088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:14.950119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7d37a-5135-4b23-82a6-0ef708565088 service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:14.950444 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7d37a-5135-4b23-82a6-0ef708565088 service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:14.950714 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7d37a-5135-4b23-82a6-0ef708565088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:15.037503 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7d37a-5135-4b23-82a6-0ef708565088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:15.039870 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7d37a-5135-4b23-82a6-0ef708565088 service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:15.049524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1396/2780] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:16.590433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:16.591881 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:16.655920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b94c6d-25fa-4b7e-9561-848a4b85928a service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:16.660770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:16.661401 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:16.661679 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:16.661949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] PATH_INFO: `/v3/users/1192cc9ef0e840c187897a9359ebee14` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:16.663211 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1192cc9ef0e840c187897a9359ebee14)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:16.665166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:16.726379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1192cc9ef0e840c187897a9359ebee14'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:16.727037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1192cc9ef0e840c187897a9359ebee14'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:16.738733 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1192cc9ef0e840c187897a9359ebee14'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:16.746467 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1192cc9ef0e840c187897a9359ebee14'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:16.760926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1192cc9ef0e840c187897a9359ebee14'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:16.768753 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1385/2781] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:16 2022] DELETE /identity/v3/users/1192cc9ef0e840c187897a9359ebee14 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:16.773695 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:16.774230 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:16.774360 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:16.795840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:16.797178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:16.837351 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:16.858698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1397/2782] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:16.869044 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:16.870804 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:16.886596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:16.933014 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf58b6-d729-4442-9acf-7b2b17294405 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:16.936826 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cdbd9-b44d-4f63-a544-204196081e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:16.937414 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdbd9-b44d-4f63-a544-204196081e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:16.937691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdbd9-b44d-4f63-a544-204196081e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:16.937957 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdbd9-b44d-4f63-a544-204196081e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:16.946901 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cdbd9-b44d-4f63-a544-204196081e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:16.948306 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cdbd9-b44d-4f63-a544-204196081e81 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:16.957735 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1386/2783] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:17.255149 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.256815 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.269769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94502b2-d783-4086-a4c5-c784ffa2fa47 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.275211 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.275908 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.276190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.276464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] PATH_INFO: `/v3/projects/9fbabfa2e2734fc099a648662ec9472d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.277774 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fbabfa2e2734fc099a648662ec9472d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.279951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.327385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fbabfa2e2734fc099a648662ec9472d', 'request_id': 'req-96c17f2e-3be2-4542-8241-fd21aaef6ad9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.335817 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fbabfa2e2734fc099a648662ec9472d', 'request_id': 'req-96c17f2e-3be2-4542-8241-fd21aaef6ad9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.342058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fbabfa2e2734fc099a648662ec9472d', 'request_id': 'req-96c17f2e-3be2-4542-8241-fd21aaef6ad9'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.356030 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] The token cache is being invalidate because project 9fbabfa2e2734fc099a648662ec9472d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:17.356319 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.357035 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-96c17f2e-3be2-4542-8241-fd21aaef6ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.358681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1398/2784] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:17 2022] DELETE /identity/v3/projects/9fbabfa2e2734fc099a648662ec9472d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:17.364437 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.364675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.364947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.385589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:17.386832 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.456535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1387/2785] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:17.464536 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.464818 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.465092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.484686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:17.485899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.494797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1399/2786] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:17.500022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.515519 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021951-c68b-4e3e-8f2e-646f993c8cb9 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.520934 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.521564 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.521841 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.522112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.522825 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:49:17.523150 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.524898 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.536451 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1388/2787] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:49:17.540803 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.550070 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd9b6b-1928-4396-987c-66d1871c74cb None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.555165 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.555872 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.556151 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.556423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.557045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.558984 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.587945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1400/2788] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:17 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:17.593700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.604537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a60b6-43a6-4c5c-9ab9-273e2587bbb5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.646345 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.648142 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.664534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.665223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.665502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.665776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.666441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.668360 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.691726 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1389/2789] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:17 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 02:49:17.695586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.705232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0e649-cb86-4bb0-86c7-d53215e219a5 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.709931 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.710557 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.710844 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.711130 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.711850 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.713441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.714785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69bb5d-e7e2-49db-b039-fd08ca51b0ae admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.719382 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.720012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.720291 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.720563 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] PATH_INFO: `/v3/users/fc0fac5eee834b30a5703d3cfeb96732` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.721743 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc0fac5eee834b30a5703d3cfeb96732)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.723702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.724106 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1390/2790] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:17.727994 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.738534 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ab57-d2c7-44ee-83b3-9db0bb321439 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.743920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.744533 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.744813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.745084 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.745789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.747380 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.757289 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1391/2791] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:49:17.761445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.771605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d1efd-3db2-47f9-893a-346cfae1cbab admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.772340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:17.776776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db1994-22fe-4e64-a4eb-4de506c9d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.777422 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db1994-22fe-4e64-a4eb-4de506c9d261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.777698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db1994-22fe-4e64-a4eb-4de506c9d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.777972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db1994-22fe-4e64-a4eb-4de506c9d261 admin admin] PATH_INFO: `/v3/projects/1517fba770884d69bbb009ccd44ef5c1/users/58987a9914554296b46e06075be5285f/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.778795 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc0fac5eee834b30a5703d3cfeb96732'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.781500 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:17.787392 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc0fac5eee834b30a5703d3cfeb96732'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.788337 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc0fac5eee834b30a5703d3cfeb96732'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.794447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db1994-22fe-4e64-a4eb-4de506c9d261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1517fba770884d69bbb009ccd44ef5c1, user_id=58987a9914554296b46e06075be5285f, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.795809 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc0fac5eee834b30a5703d3cfeb96732'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.797118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db1994-22fe-4e64-a4eb-4de506c9d261 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.808309 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-673fa18f-c225-4d30-84e6-8065fa7cb325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc0fac5eee834b30a5703d3cfeb96732'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:17.811033 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1392/2792] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:17 2022] PUT /identity/v3/projects/1517fba770884d69bbb009ccd44ef5c1/users/58987a9914554296b46e06075be5285f/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:49:17.814325 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1401/2793] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:17 2022] DELETE /identity/v3/users/fc0fac5eee834b30a5703d3cfeb96732 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:49:17.818723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.819030 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.819305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.822315 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.822603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.822874 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.838208 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:17.839431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.849747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] MFA Rules not processed for user `58987a9914554296b46e06075be5285f`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:17.851128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.900942 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1393/2794] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:49:17.910947 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.912355 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.916606 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1402/2795] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:49:17.927299 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:17.928792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:17.968541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd4b55-1280-4255-9533-1ec5ffc9968a None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.972811 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.973573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.973851 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.974123 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.983737 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.984463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce5fa7-4e64-4b9d-85e1-abc9483df493 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:17.985175 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:17.988446 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:17.989056 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:17.989371 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:17.989643 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:17.995305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1394/2796] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:49:17.998127 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:17.999610 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:18.008792 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1403/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:18.288972 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:18.290695 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:18.314005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97fdb8-d1b3-45e7-ae63-5553926edfb9 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:18.319899 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:18.320550 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:18.328904 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:18.334654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] PATH_INFO: `/v3/projects/b57099246b2242cd833e6fdb505079b6` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:18.336067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57099246b2242cd833e6fdb505079b6)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:18.338348 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:18.395546 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57099246b2242cd833e6fdb505079b6', 'request_id': 'req-bd0b1b36-ccc2-4687-af19-423dd9b894b2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.403147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57099246b2242cd833e6fdb505079b6', 'request_id': 'req-bd0b1b36-ccc2-4687-af19-423dd9b894b2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.409617 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57099246b2242cd833e6fdb505079b6', 'request_id': 'req-bd0b1b36-ccc2-4687-af19-423dd9b894b2'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.425145 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] The token cache is being invalidate because project b57099246b2242cd833e6fdb505079b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:18.425443 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.426086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.427655 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1395/2798] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:18 2022] DELETE /identity/v3/projects/b57099246b2242cd833e6fdb505079b6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:18.431702 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:18.442699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948c67-8149-4330-aa39-a4fd7f039ccb service cinder] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:18.503067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:18.517225 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:18.517871 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:18.518147 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:18.518458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] PATH_INFO: `/v3/users/69812b6193e64c3581d5da6d6f1ddf2c` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:18.519598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69812b6193e64c3581d5da6d6f1ddf2c)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:18.521652 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:18.569934 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69812b6193e64c3581d5da6d6f1ddf2c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.578055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69812b6193e64c3581d5da6d6f1ddf2c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.578769 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69812b6193e64c3581d5da6d6f1ddf2c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.586413 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69812b6193e64c3581d5da6d6f1ddf2c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.599100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-c4c05ec5-75d8-4d77-9b8d-1ff895bca985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69812b6193e64c3581d5da6d6f1ddf2c'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:18.605192 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1404/2799] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:18 2022] DELETE /identity/v3/users/69812b6193e64c3581d5da6d6f1ddf2c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:18.879177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:18.880663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:18.894379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b1b36-ccc2-4687-af19-423dd9b894b2 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:18.903834 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:18.954600 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:18.955152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:18.955431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:18.955704 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] PATH_INFO: `/v3/projects/260c9333214441eea6f39c69aef4b071` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:18.956824 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=260c9333214441eea6f39c69aef4b071)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:18.958789 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.010171 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260c9333214441eea6f39c69aef4b071', 'request_id': 'req-b7616760-90d6-4d34-aa50-eb3a444de5de'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:19.016544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260c9333214441eea6f39c69aef4b071', 'request_id': 'req-b7616760-90d6-4d34-aa50-eb3a444de5de'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:19.022431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260c9333214441eea6f39c69aef4b071', 'request_id': 'req-b7616760-90d6-4d34-aa50-eb3a444de5de'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:19.036353 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] The token cache is being invalidate because project 260c9333214441eea6f39c69aef4b071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:19.036641 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:19.037251 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-b7616760-90d6-4d34-aa50-eb3a444de5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:19.038792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1396/2800] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:18 2022] DELETE /identity/v3/projects/260c9333214441eea6f39c69aef4b071 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:19.043783 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.044260 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.044389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.065869 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:19.067103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:19.129202 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1405/2801] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:49:19.142296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.142630 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.142986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.166041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:19.167290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:19.176081 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1397/2802] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:19.180524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.190421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef03b2b-f3a8-4534-a45a-28930d64aff1 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.195527 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.196157 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.196431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.196700 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.197362 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:49:19.197663 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.199320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.211215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1406/2803] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:49:19.215718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.225077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba388-90af-4ddd-bf3f-f0842199f19c None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.230750 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.231350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.231626 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.231895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.232515 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.234427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.265050 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1398/2804] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:19.269385 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.280658 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa18142-3296-4a6a-9baf-21a5bb40395c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.331463 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:19.341177 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.342100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.342588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.343198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.343881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.345802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.370734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1407/2805] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:19.375224 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.386140 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864668-f71a-4f52-bd7a-16d1aeba3b54 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.392116 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.392728 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.393009 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.393284 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.393937 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.395595 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.406032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1399/2806] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:19.410229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.421614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed2dae-8952-48a1-b582-a19115a1de2c admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.427067 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a92e-ecbf-4b76-a058-f3fd85062121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.427823 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a92e-ecbf-4b76-a058-f3fd85062121 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.428103 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a92e-ecbf-4b76-a058-f3fd85062121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.428363 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a92e-ecbf-4b76-a058-f3fd85062121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.429024 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7a92e-ecbf-4b76-a058-f3fd85062121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.430668 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7a92e-ecbf-4b76-a058-f3fd85062121 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.442217 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1408/2807] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:19.446817 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.457372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981168c8-9a9c-4991-828b-9eb47e8f7a73 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.463166 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.463770 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.464052 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.464323 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] PATH_INFO: `/v3/projects/1193551247914cf387fe23c3ff89ab6d/users/d4416210564546c986526f8d04fd4c6a/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.480556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1193551247914cf387fe23c3ff89ab6d, user_id=d4416210564546c986526f8d04fd4c6a, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.483283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.503158 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1400/2808] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:19 2022] PUT /identity/v3/projects/1193551247914cf387fe23c3ff89ab6d/users/d4416210564546c986526f8d04fd4c6a/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:49:19.513641 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.513926 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.514224 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.543013 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] MFA Rules not processed for user `d4416210564546c986526f8d04fd4c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:19.544339 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:19.610797 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1409/2809] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:19.615614 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.656270 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:19.678061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d0cee-1e15-4c36-a6ba-68adcb611db8 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.683516 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.684264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.684547 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.684830 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.685445 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.687485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.716681 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1401/2810] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:19.721117 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.732535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22cba-b74c-4fd5-a1a5-bdc427167693 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.801821 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.802715 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.803193 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.803614 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.804734 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.808538 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.833394 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1410/2811] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:19 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:19.834694 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.845126 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d73e2-f500-4d0e-adaa-a30dbdfe47a7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.849823 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.850461 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.850742 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.851013 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.851664 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.853287 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.863874 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1402/2812] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 02:49:19.868053 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.878854 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192861ef-8240-4b9c-988e-b4b57ddb1a34 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.884601 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.885252 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.885531 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.885799 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] PATH_INFO: `/v3/projects/9cfb9f5948404f64b2a100e38392b325/users/1fb60dd3f4c3481b9f3db5018e95050a/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.901851 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cfb9f5948404f64b2a100e38392b325, user_id=1fb60dd3f4c3481b9f3db5018e95050a, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.904636 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:19.925468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:19.925917 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1411/2813] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:19 2022] PUT /identity/v3/projects/9cfb9f5948404f64b2a100e38392b325/users/1fb60dd3f4c3481b9f3db5018e95050a/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:49:19.936466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b8b6a-6969-4fbc-a4ed-66343b8c9911 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:19.992945 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3311ee-2956-46c8-bf39-a32d14e7ccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:19.993501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3311ee-2956-46c8-bf39-a32d14e7ccc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:19.993785 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3311ee-2956-46c8-bf39-a32d14e7ccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:19.994057 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3311ee-2956-46c8-bf39-a32d14e7ccc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:19.994738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3311ee-2956-46c8-bf39-a32d14e7ccc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:19.996301 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3311ee-2956-46c8-bf39-a32d14e7ccc7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:20.005812 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1403/2814] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:20.010244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:20.020822 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d4375-83d2-4384-a943-4a4fbb3aeabf admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:20.025777 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:20.026473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:20.026767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:20.027020 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] PATH_INFO: `/v3/projects/9cfb9f5948404f64b2a100e38392b325/users/1fb60dd3f4c3481b9f3db5018e95050a/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:20.029005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cfb9f5948404f64b2a100e38392b325, user_id=1fb60dd3f4c3481b9f3db5018e95050a, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:20.031687 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:20.042093 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1412/2815] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:20 2022] PUT /identity/v3/projects/9cfb9f5948404f64b2a100e38392b325/users/1fb60dd3f4c3481b9f3db5018e95050a/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:20.057548 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:20.057839 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:20.058124 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:20.086675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] MFA Rules not processed for user `1fb60dd3f4c3481b9f3db5018e95050a`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:20.087873 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:20.166206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1404/2816] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:20 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:20.177429 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:20.179428 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:20.200458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:20.239234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175fe5f-d695-40e6-af92-ee58d7ef37f0 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:20.243367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538d195-1299-4af2-a808-48d17c364a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:20.243899 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538d195-1299-4af2-a808-48d17c364a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:20.244178 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538d195-1299-4af2-a808-48d17c364a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:20.244432 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538d195-1299-4af2-a808-48d17c364a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:20.253499 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538d195-1299-4af2-a808-48d17c364a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:20.254973 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538d195-1299-4af2-a808-48d17c364a0a service cinder] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:20.264388 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1413/2817] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:20 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:20.405236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:20.443813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:20.476577 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bed032-1c16-4d4e-9977-37fa46a5d0a5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:20.481629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:20.482304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:20.482586 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:20.482863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] PATH_INFO: `/v3/users/d4416210564546c986526f8d04fd4c6a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:20.484021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4416210564546c986526f8d04fd4c6a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:20.485952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:20.534891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4416210564546c986526f8d04fd4c6a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:20.535556 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4416210564546c986526f8d04fd4c6a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:20.544192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4416210564546c986526f8d04fd4c6a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:20.551609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4416210564546c986526f8d04fd4c6a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:20.563432 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4416210564546c986526f8d04fd4c6a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:20.569244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1405/2818] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:20 2022] DELETE /identity/v3/users/d4416210564546c986526f8d04fd4c6a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:20.573894 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:20.574185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:20.574485 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:20.593509 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:20.594754 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:20.654420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1414/2819] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:49:20.664464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:20.666095 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:20.727192 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e713-0244-4043-a9d7-f3575ca681ef admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:20.731840 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:20.732447 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:20.732725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:20.732997 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:20.742998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:20.744468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:20.754181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1406/2820] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:21.040199 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:21.041748 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:21.056747 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e122-e850-4033-b30d-d1952a686986 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:21.063261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:21.064005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.064324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.064554 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] PATH_INFO: `/v3/projects/1193551247914cf387fe23c3ff89ab6d` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.065829 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1193551247914cf387fe23c3ff89ab6d)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:21.068009 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:21.117109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1193551247914cf387fe23c3ff89ab6d', 'request_id': 'req-a96144b1-7cad-4eda-b16f-a1a6f1806657'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.134939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1193551247914cf387fe23c3ff89ab6d', 'request_id': 'req-a96144b1-7cad-4eda-b16f-a1a6f1806657'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.142423 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1193551247914cf387fe23c3ff89ab6d', 'request_id': 'req-a96144b1-7cad-4eda-b16f-a1a6f1806657'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.154234 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] The token cache is being invalidate because project 1193551247914cf387fe23c3ff89ab6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:21.154530 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.155322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.156898 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1415/2821] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:21 2022] DELETE /identity/v3/projects/1193551247914cf387fe23c3ff89ab6d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:21.160854 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:21.170722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89ee1d-7985-401f-8803-e0691d39e861 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:21.223721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:21.232188 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:21.232863 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.233290 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.233725 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] PATH_INFO: `/v3/users/1fb60dd3f4c3481b9f3db5018e95050a` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.235041 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb60dd3f4c3481b9f3db5018e95050a)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:21.237004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:21.286693 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb60dd3f4c3481b9f3db5018e95050a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.287285 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb60dd3f4c3481b9f3db5018e95050a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.296403 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb60dd3f4c3481b9f3db5018e95050a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.303560 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb60dd3f4c3481b9f3db5018e95050a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.315948 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a99e9c0b-57d4-489c-982a-2440921819f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb60dd3f4c3481b9f3db5018e95050a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.322296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1407/2822] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:21 2022] DELETE /identity/v3/users/1fb60dd3f4c3481b9f3db5018e95050a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:21.596082 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:21.597660 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:21.614620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96144b1-7cad-4eda-b16f-a1a6f1806657 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:21.657615 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:21.673473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:21.674114 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.674458 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.674729 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] PATH_INFO: `/v3/projects/9cfb9f5948404f64b2a100e38392b325` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.675834 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cfb9f5948404f64b2a100e38392b325)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:21.677745 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:21.725464 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cfb9f5948404f64b2a100e38392b325', 'request_id': 'req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.734349 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cfb9f5948404f64b2a100e38392b325', 'request_id': 'req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.741278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cfb9f5948404f64b2a100e38392b325', 'request_id': 'req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.752533 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] The token cache is being invalidate because project 9cfb9f5948404f64b2a100e38392b325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:21.752801 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.753479 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-95d0a8f0-bb76-409c-b82a-6ef5f28ace42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:21.755040 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1416/2823] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:21 2022] DELETE /identity/v3/projects/9cfb9f5948404f64b2a100e38392b325 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:21.760223 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.760512 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.760858 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.780246 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:21.781439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:21.840398 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1408/2824] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:21.852683 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.853064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.853353 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.880043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:21.881289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:21.891688 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1417/2825] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:21.897031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:21.916444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fe4dc-b071-4832-b094-3426be8d1322 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:21.923113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:21.923723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.923999 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.924260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.925037 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:49:21.925349 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:21.927004 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:21.939497 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1409/2826] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 02:49:21.946200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:21.960791 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b14f9-869e-4853-b3a5-80dd39407720 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:21.968041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:21.968639 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:21.968916 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:21.969182 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:21.969802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:21.971773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.002703 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1418/2827] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:21 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:22.006888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.016935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c50fa0-26b3-478e-ba4b-206a0df169a7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.075976 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.076805 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.077189 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.077551 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.078583 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.081667 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.092961 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:22.106212 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1410/2828] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:22 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:22.110426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.121473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389c20-9d46-4f00-afe4-091140928c41 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.127265 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.127891 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.128166 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.128431 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.129077 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.130723 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.141448 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1419/2829] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:22.145217 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.156101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85456ca1-bd5e-44a2-b19b-e050187a9b54 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.161570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46188ccf-499e-43df-a2be-7b846df50e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.162221 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46188ccf-499e-43df-a2be-7b846df50e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.162464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46188ccf-499e-43df-a2be-7b846df50e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.162734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46188ccf-499e-43df-a2be-7b846df50e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.163370 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46188ccf-499e-43df-a2be-7b846df50e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.164998 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46188ccf-499e-43df-a2be-7b846df50e47 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.175815 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1411/2830] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:22.179966 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.190621 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16ddab-009c-49bd-9961-c16696990f55 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.196914 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.197608 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] REQUEST_METHOD: `PUT` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.197859 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.198414 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] PATH_INFO: `/v3/projects/1d9235eb09ec4b878421f9e9d80d158c/users/e9b6074248ce4a6f825d6ddf2198b67b/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.213198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9235eb09ec4b878421f9e9d80d158c, user_id=e9b6074248ce4a6f825d6ddf2198b67b, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.215927 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.230681 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1420/2831] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:22 2022] PUT /identity/v3/projects/1d9235eb09ec4b878421f9e9d80d158c/users/e9b6074248ce4a6f825d6ddf2198b67b/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:22.241717 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.242007 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.242315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.271101 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] MFA Rules not processed for user `e9b6074248ce4a6f825d6ddf2198b67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:22.272243 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:22.350501 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1412/2832] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:22.355119 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.423080 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15efa5a-ba62-4a1a-a502-537fd517daba admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.428235 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.428853 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.429135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.429384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.430043 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.432005 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.449556 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:22.465985 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1421/2833] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:22 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:22.467177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.478392 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba6d6d-997f-4e36-93be-bacd1d805429 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.537283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.537905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.538276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.538562 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] PATH_INFO: `/v3/users` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.539191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.541162 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.563174 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1413/2834] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:22 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:22.567037 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.577686 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ae10d-ba38-4dce-b08a-bbe067616d02 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.582598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.583289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.583638 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.584038 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.584833 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.586649 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.601133 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1422/2835] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:22.603008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.614093 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cde63-9c05-4b7a-bc19-7a4eba14ad48 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.618980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.619637 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.619916 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.620264 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] PATH_INFO: `/v3/projects/9e855abbab484eaa8974a15379b55152/users/25ab10d8a6c940aa93babfc207108070/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.635895 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e855abbab484eaa8974a15379b55152, user_id=25ab10d8a6c940aa93babfc207108070, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.638634 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.654020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1414/2836] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:22 2022] PUT /identity/v3/projects/9e855abbab484eaa8974a15379b55152/users/25ab10d8a6c940aa93babfc207108070/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:22.658244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.668242 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72a65a-1b8f-4a2f-b4eb-bedee3ef3b8d admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.727378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1b1de-c10c-4846-978f-1e6e0b307035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.727994 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1b1de-c10c-4846-978f-1e6e0b307035 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.728273 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1b1de-c10c-4846-978f-1e6e0b307035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.728588 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1b1de-c10c-4846-978f-1e6e0b307035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.729231 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1b1de-c10c-4846-978f-1e6e0b307035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.730892 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1b1de-c10c-4846-978f-1e6e0b307035 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.740976 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1423/2837] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:22.745336 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.758739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4ad0-2ff9-43a8-8bf7-85053f4f3e5e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:22.764006 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:22.764654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.764949 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.765197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] PATH_INFO: `/v3/projects/9e855abbab484eaa8974a15379b55152/users/25ab10d8a6c940aa93babfc207108070/roles/4483761ca5a0414abb0dff9144ea2ce5` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.767245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e855abbab484eaa8974a15379b55152, user_id=25ab10d8a6c940aa93babfc207108070, role_id=4483761ca5a0414abb0dff9144ea2ce5)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:22.769913 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:22.781092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1415/2838] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:22 2022] PUT /identity/v3/projects/9e855abbab484eaa8974a15379b55152/users/25ab10d8a6c940aa93babfc207108070/roles/4483761ca5a0414abb0dff9144ea2ce5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:22.850250 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:22.850548 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:22.850828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:22.880239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] MFA Rules not processed for user `25ab10d8a6c940aa93babfc207108070`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:22.881460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:22.953591 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1424/2839] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:22.964853 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:22.966747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:22.994529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:23.028056 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a18e-d37e-44b5-9f0f-539fb56f3f9a admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:23.032085 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151991b7-440c-49a9-a581-804066eb06af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:23.032700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151991b7-440c-49a9-a581-804066eb06af service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:23.032981 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151991b7-440c-49a9-a581-804066eb06af service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:23.033234 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151991b7-440c-49a9-a581-804066eb06af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:23.042644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151991b7-440c-49a9-a581-804066eb06af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:23.044201 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151991b7-440c-49a9-a581-804066eb06af service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:23.054256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1416/2840] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 02:49:23.443278 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:23.514947 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c43ae-5ba3-4945-9102-4f39f32e3e8a None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:23.521421 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:23.522060 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:23.522378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:23.522655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] PATH_INFO: `/v3/users/e9b6074248ce4a6f825d6ddf2198b67b` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:23.523826 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b6074248ce4a6f825d6ddf2198b67b)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:23.525758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:23.532375 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=100 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:23.577773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b6074248ce4a6f825d6ddf2198b67b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:23.585758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b6074248ce4a6f825d6ddf2198b67b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:23.586625 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b6074248ce4a6f825d6ddf2198b67b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:23.593920 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b6074248ce4a6f825d6ddf2198b67b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:23.606258 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b6074248ce4a6f825d6ddf2198b67b'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:23.613065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1425/2841] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:23 2022] DELETE /identity/v3/users/e9b6074248ce4a6f825d6ddf2198b67b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:23.617607 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:23.618067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:23.618181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:23.637579 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:23.638810 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:23.702535 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1417/2842] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:23.713346 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:23.714911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:23.776928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295942e-f2f1-4d34-bff7-99f819066ff4 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:23.780958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:23.781717 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:23.782041 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:23.782350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:23.792391 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:23.793857 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:23.804648 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1426/2843] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:24.083635 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:24.092536 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:24.104541 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f672a-c2bd-41ba-8f92-88344bb20e60 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:24.109344 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:24.110070 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:24.110482 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:24.110821 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] PATH_INFO: `/v3/projects/1d9235eb09ec4b878421f9e9d80d158c` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:24.112021 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9235eb09ec4b878421f9e9d80d158c)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:24.115304 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:24.171891 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9235eb09ec4b878421f9e9d80d158c', 'request_id': 'req-a279d88b-8031-4c23-92fe-303ef298a26d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.178253 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9235eb09ec4b878421f9e9d80d158c', 'request_id': 'req-a279d88b-8031-4c23-92fe-303ef298a26d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.183995 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9235eb09ec4b878421f9e9d80d158c', 'request_id': 'req-a279d88b-8031-4c23-92fe-303ef298a26d'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.196783 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] The token cache is being invalidate because project 1d9235eb09ec4b878421f9e9d80d158c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:24.197077 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.197702 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.199485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1418/2844] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:24 2022] DELETE /identity/v3/projects/1d9235eb09ec4b878421f9e9d80d158c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:24.203635 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:24.213494 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f60276-29c8-45f4-998d-127737cb6e79 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:24.272044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:24.272682 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:24.272962 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:24.273239 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] PATH_INFO: `/v3/users/25ab10d8a6c940aa93babfc207108070` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:24.274434 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ab10d8a6c940aa93babfc207108070)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:24.276476 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:24.315877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:24.332699 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ab10d8a6c940aa93babfc207108070'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.340945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ab10d8a6c940aa93babfc207108070'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.341612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ab10d8a6c940aa93babfc207108070'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.349011 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ab10d8a6c940aa93babfc207108070'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.361518 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-d6dd7297-d982-4b42-af1c-1120b2d81805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ab10d8a6c940aa93babfc207108070'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.368111 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1427/2845] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:24 2022] DELETE /identity/v3/users/25ab10d8a6c940aa93babfc207108070 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:24.673689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:24.676071 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:24.691494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d88b-8031-4c23-92fe-303ef298a26d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:24.739485 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:24.751920 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:24.752552 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:24.752827 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:24.753097 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] PATH_INFO: `/v3/projects/9e855abbab484eaa8974a15379b55152` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:24.754236 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e855abbab484eaa8974a15379b55152)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:24.756139 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:24.805305 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e855abbab484eaa8974a15379b55152', 'request_id': 'req-2adee1ee-b1f8-4454-aacc-6118cdc21acf'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.811636 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e855abbab484eaa8974a15379b55152', 'request_id': 'req-2adee1ee-b1f8-4454-aacc-6118cdc21acf'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.820718 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e855abbab484eaa8974a15379b55152', 'request_id': 'req-2adee1ee-b1f8-4454-aacc-6118cdc21acf'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.842514 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] The token cache is being invalidate because project 9e855abbab484eaa8974a15379b55152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:24.842842 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.843686 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-2adee1ee-b1f8-4454-aacc-6118cdc21acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:24.845444 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1419/2846] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:24 2022] DELETE /identity/v3/projects/9e855abbab484eaa8974a15379b55152 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:24.851443 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:24.851979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:24.851979 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:24.877603 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:24.878880 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:24.939215 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1428/2847] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:24.946993 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:24.947283 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:24.947584 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:24.968610 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:24.970011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:24.978944 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1420/2848] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:24.983373 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:24.994367 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66e680-9807-4fc7-ba4c-cade21f87ea2 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:24.999050 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:24.999667 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:24.999951 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.000222 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.000904 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:49:25.001210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.002875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.015064 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1429/2849] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:49:25.020232 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:25.029958 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051b6b0-99ec-4d85-83f5-8d0445f9df31 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:25.035409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:25.036147 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.036427 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.036700 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.037354 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.039307 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.068181 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1421/2850] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:49:25 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:25.072487 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:25.082986 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae0856-5578-4284-a2b0-aa029859acdc admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:25.140672 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:25.141286 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.141580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.141848 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.142507 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.144454 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.167361 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1430/2851] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:49:25 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:49:25.171474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:25.181952 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686b1d3-dfb6-4d7f-b01d-48ed586bef9a admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:25.187590 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:25.188191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.188508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.188747 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.189455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.191092 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.201654 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1422/2852] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:25.205680 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:25.216039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69c513-1de4-440a-9ee0-0fc8db4fd14f admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:25.221340 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a702ba-efab-4e99-9293-2e9eef796be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:25.221936 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a702ba-efab-4e99-9293-2e9eef796be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.222350 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a702ba-efab-4e99-9293-2e9eef796be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.222624 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a702ba-efab-4e99-9293-2e9eef796be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.223268 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a702ba-efab-4e99-9293-2e9eef796be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.224862 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a702ba-efab-4e99-9293-2e9eef796be7 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.236069 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1431/2853] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:49:25.240727 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:25.250974 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586523f-b8ba-425c-a94e-3455b56bb55e admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:25.256169 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:25.256808 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.257091 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.257356 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] PATH_INFO: `/v3/projects/1ac907637c064b2499900b106ea18ab0/users/49614df61d1e4c4ebe333db2017e9636/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.272207 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac907637c064b2499900b106ea18ab0, user_id=49614df61d1e4c4ebe333db2017e9636, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.274896 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.291369 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1423/2854] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:49:25 2022] PUT /identity/v3/projects/1ac907637c064b2499900b106ea18ab0/users/49614df61d1e4c4ebe333db2017e9636/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:25.296767 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7c354-da03-48c5-97d9-59c5902c6eb4 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.297039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7c354-da03-48c5-97d9-59c5902c6eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.297314 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7c354-da03-48c5-97d9-59c5902c6eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.315909 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-71e7c354-da03-48c5-97d9-59c5902c6eb4 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:25.317113 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-71e7c354-da03-48c5-97d9-59c5902c6eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:25.326065 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:25.383607 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1432/2855] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:25.397450 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:25.400670 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:25.460692 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e168c97-3a3d-47a9-9712-e5a5ebbf6f3d admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:25.464792 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:25.465408 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:25.465685 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:25.465953 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:25.475508 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:25.476933 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:25.487011 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1424/2856] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:26.165662 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f395a-8969-42d8-9263-73ae0d26d9eb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:26.166221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f395a-8969-42d8-9263-73ae0d26d9eb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:26.166221 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f395a-8969-42d8-9263-73ae0d26d9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:26.195279 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-bf2f395a-8969-42d8-9263-73ae0d26d9eb None None] MFA Rules not processed for user `49614df61d1e4c4ebe333db2017e9636`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:26.196768 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-bf2f395a-8969-42d8-9263-73ae0d26d9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:26.262975 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1433/2857] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:26.274008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:26.275622 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:26.297989 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:26.336736 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9175399-7c45-462a-8c97-8925575b0c38 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:26.341802 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:26.342455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:26.342734 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:26.343005 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:26.352035 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:26.353466 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:26.363022 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1425/2858] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:27.744604 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfcae7-c806-418c-882a-c6f69453bb76 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:27.745185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfcae7-c806-418c-882a-c6f69453bb76 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:27.745185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfcae7-c806-418c-882a-c6f69453bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:27.766259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-95bfcae7-c806-418c-882a-c6f69453bb76 None None] MFA Rules not processed for user `49614df61d1e4c4ebe333db2017e9636`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:27.767473 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-95bfcae7-c806-418c-882a-c6f69453bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:27.775655 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1434/2859] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:28.032867 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:28.034474 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:28.048744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c0c96-1e71-42ce-8e85-c111a8ad97ed service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:28.053113 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:28.053739 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:28.054017 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:28.054317 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] PATH_INFO: `/v3/users/58987a9914554296b46e06075be5285f` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:28.055441 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58987a9914554296b46e06075be5285f)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:28.057364 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:28.106125 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58987a9914554296b46e06075be5285f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.106986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58987a9914554296b46e06075be5285f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.115709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58987a9914554296b46e06075be5285f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.123155 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58987a9914554296b46e06075be5285f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.135177 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58987a9914554296b46e06075be5285f'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.141400 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1426/2860] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:49:28 2022] DELETE /identity/v3/users/58987a9914554296b46e06075be5285f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:28.146746 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:28.146983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:28.147289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:28.171389 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:28.173368 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:28.240952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1435/2861] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 02:49:28.251118 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:28.252751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:28.315528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22659a81-576e-48bd-ac1d-007aa1e11c53 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:28.319788 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d005c-3188-4bd5-8093-11f8acae9e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:28.320429 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d005c-3188-4bd5-8093-11f8acae9e52 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:28.320706 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d005c-3188-4bd5-8093-11f8acae9e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:28.320984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d005c-3188-4bd5-8093-11f8acae9e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:28.331476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d005c-3188-4bd5-8093-11f8acae9e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:28.332935 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d005c-3188-4bd5-8093-11f8acae9e52 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:28.343494 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1427/2862] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:49:28.646324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:49:28.648742 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:28.664766 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34ad8-81a0-488f-842a-3edad1bbe44d None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:49:28.676158 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:49:28.677170 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:28.677620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:28.678416 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] PATH_INFO: `/v3/projects/1517fba770884d69bbb009ccd44ef5c1` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:28.679758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1517fba770884d69bbb009ccd44ef5c1)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:49:28.681945 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:49:28.720525 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:28.735083 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1517fba770884d69bbb009ccd44ef5c1', 'request_id': 'req-2457fb1c-8bfa-4808-acef-a991da04c247'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.745600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1517fba770884d69bbb009ccd44ef5c1', 'request_id': 'req-2457fb1c-8bfa-4808-acef-a991da04c247'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.752255 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1517fba770884d69bbb009ccd44ef5c1', 'request_id': 'req-2457fb1c-8bfa-4808-acef-a991da04c247'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.762905 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] The token cache is being invalidate because project 1517fba770884d69bbb009ccd44ef5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:49:28.763191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.763911 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-2457fb1c-8bfa-4808-acef-a991da04c247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:49:28.765596 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1436/2863] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:49:28 2022] DELETE /identity/v3/projects/1517fba770884d69bbb009ccd44ef5c1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:49:38.338964 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=73,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:38.759676 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:49:43.491922 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:49:43.492455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:49:43.492455 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:49:43.511905 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] MFA Rules not processed for user `49614df61d1e4c4ebe333db2017e9636`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:49:43.513151 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:49:43.571744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1428/2864] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:49:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:49:53.564956 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=25 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:02.765277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa94b7-2527-4a2d-bd89-72567952b57b None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:02.765785 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa94b7-2527-4a2d-bd89-72567952b57b None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:02.765903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa94b7-2527-4a2d-bd89-72567952b57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:02.795946 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-e4fa94b7-2527-4a2d-bd89-72567952b57b None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:02.797200 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-e4fa94b7-2527-4a2d-bd89-72567952b57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:02.857163 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1437/2865] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 9 02:50:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:02.907676 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:02.909197 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:02.991385 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da448-0de1-44e0-a356-4458d5c176ad None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:02.997362 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:02.997984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:02.998294 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:02.998572 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:03.007096 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:03.008677 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:03.016938 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1429/2866] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:50:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:50:12.850322 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=32 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:13.013782 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=32 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:18.542455 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:18.542963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:18.542963 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:18.564335 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] MFA Rules not processed for user `562c6203e9cf440ebd81d14877c9e452`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:18.565589 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:18.573758 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1438/2867] 10.222.0.20 () {58 vars in 967 bytes} [Tue Aug 9 02:50:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:18.587238 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:18.599072 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f8402-0634-4945-a51c-ff034f299e2c service swift] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:18.605721 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea3e582aac94931b1a041817ccab685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:18.606411 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:18.606689 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:18.606960 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:18.616820 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:18.618472 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:18.627691 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1430/2868] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 9 02:50:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 02:50:20.630284 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:20.631875 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:20.702183 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a9404-21c3-4d31-bd23-06aa3fda99b3 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:20.707210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:20.707847 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:20.708129 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:20.708403 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] PATH_INFO: `/v3/users/49614df61d1e4c4ebe333db2017e9636` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:20.709602 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49614df61d1e4c4ebe333db2017e9636)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:20.711581 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:20.758952 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49614df61d1e4c4ebe333db2017e9636'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:20.766523 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49614df61d1e4c4ebe333db2017e9636'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:20.767144 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49614df61d1e4c4ebe333db2017e9636'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:20.774102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49614df61d1e4c4ebe333db2017e9636'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:20.786130 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-15f170db-3afc-4d50-8333-ee87f7a9d754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49614df61d1e4c4ebe333db2017e9636'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:20.792036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1439/2869] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:50:20 2022] DELETE /identity/v3/users/49614df61d1e4c4ebe333db2017e9636 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:50:20.968456 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:20.980431 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c334ab-90e3-4055-b686-2ceb729f9384 service swift] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.041198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.041767 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.042043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.042342 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] PATH_INFO: `/v3/projects/1ac907637c064b2499900b106ea18ab0` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.043596 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac907637c064b2499900b106ea18ab0)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.045504 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.094865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac907637c064b2499900b106ea18ab0', 'request_id': 'req-41deb477-7e6f-490c-b91e-c985b639933a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:21.101279 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac907637c064b2499900b106ea18ab0', 'request_id': 'req-41deb477-7e6f-490c-b91e-c985b639933a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:21.107230 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac907637c064b2499900b106ea18ab0', 'request_id': 'req-41deb477-7e6f-490c-b91e-c985b639933a'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:21.121244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] The token cache is being invalidate because project 1ac907637c064b2499900b106ea18ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:50:21.121537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:21.122184 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-41deb477-7e6f-490c-b91e-c985b639933a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:21.123765 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1431/2870] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:50:20 2022] DELETE /identity/v3/projects/1ac907637c064b2499900b106ea18ab0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:50:21.128846 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.129135 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.129411 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.149270 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:21.150537 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:21.212021 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1440/2871] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:21.224315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.224642 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.224986 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.252570 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.auth.core [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:21.254529 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:21.268119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1432/2872] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:21.274386 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.285308 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a729-ed18-4ad0-9b8e-5bceb9e3fb70 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.292039 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.292789 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.293185 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.293580 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] PATH_INFO: `/v3/domains` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.294671 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 02:50:21.295128 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.297772 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.311238 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:21.315010 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1441/2873] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 9 02:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 02:50:21.320143 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.329793 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0115c-0241-4754-b8bc-48c36933beb5 None None] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.335310 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.335914 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] REQUEST_METHOD: `POST` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.336190 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.336464 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.337080 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.339049 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.378723 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1433/2874] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 9 02:50:21 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:50:21.383002 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.393055 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875204-4938-4580-8aec-0da7eb453494 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.451198 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.451814 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.452094 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.452387 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.453017 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.454995 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.478753 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1442/2875] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 9 02:50:21 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 02:50:21.483043 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.493662 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedd440-9f29-4ff7-a034-ebea2e670066 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.498191 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.498780 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.499067 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.499329 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.499969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.501585 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.511308 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1434/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:50:21.515261 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.525549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9cf15-8fef-4959-910e-a720faf67a55 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.530731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba2e66-4f7c-4f82-9a40-3f5f9d6a27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.531323 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba2e66-4f7c-4f82-9a40-3f5f9d6a27a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.531612 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba2e66-4f7c-4f82-9a40-3f5f9d6a27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.531877 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba2e66-4f7c-4f82-9a40-3f5f9d6a27a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.532567 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba2e66-4f7c-4f82-9a40-3f5f9d6a27a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.534175 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba2e66-4f7c-4f82-9a40-3f5f9d6a27a6 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.546023 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1443/2877] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 9 02:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 02:50:21.550031 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.560460 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae95f17-27c1-45b5-8e7c-4a59d8284830 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.566471 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.567138 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.567409 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.567666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] PATH_INFO: `/v3/projects/d488e2c8749640a0b3a9b0037051ff12/users/9e5f6c1a1f07409483d732db05256f1b/roles/5106c6bcdc5a482eb55acc44c11b638b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.583134 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=d488e2c8749640a0b3a9b0037051ff12, user_id=9e5f6c1a1f07409483d732db05256f1b, role_id=5106c6bcdc5a482eb55acc44c11b638b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.585773 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.601112 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1435/2878] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 9 02:50:21 2022] PUT /identity/v3/projects/d488e2c8749640a0b3a9b0037051ff12/users/9e5f6c1a1f07409483d732db05256f1b/roles/5106c6bcdc5a482eb55acc44c11b638b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:50:21.606179 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7886e-87eb-48a2-b692-5a76df50cb87 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.606460 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7886e-87eb-48a2-b692-5a76df50cb87 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.606720 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7886e-87eb-48a2-b692-5a76df50cb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.625678 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2ca7886e-87eb-48a2-b692-5a76df50cb87 None None] MFA Rules not processed for user `b1cb94a6532045fc84cecdb848b352fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:21.626903 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2ca7886e-87eb-48a2-b692-5a76df50cb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:21.686881 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1444/2879] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:21.697276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:21.699119 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:21.705738 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:21.759941 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfec2c3-0d56-490a-bdcf-c28dd8f98cde admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:21.764640 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:21.765245 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:21.765521 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:21.765772 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:21.775866 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:21.777277 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:21.787135 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1436/2880] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:50:22.419374 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbc67-eb0b-450b-8ee9-a0d3ba62e3eb None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:22.419659 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbc67-eb0b-450b-8ee9-a0d3ba62e3eb None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:22.419939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbc67-eb0b-450b-8ee9-a0d3ba62e3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:22.450773 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-0aabbc67-eb0b-450b-8ee9-a0d3ba62e3eb None None] MFA Rules not processed for user `9e5f6c1a1f07409483d732db05256f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:22.452001 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-0aabbc67-eb0b-450b-8ee9-a0d3ba62e3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:22.519906 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1445/2881] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:50:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:22.530888 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:22.532372 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:22.591771 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385515-5037-4706-9a98-2091bd278f1c service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:22.595486 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd57751a2455c48d28fcf9e97b62cfc38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:22.596098 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:22.596379 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:22.596656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:22.605510 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:22.607008 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:22.616076 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1437/2882] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:50:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:50:23.992710 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:23.993229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:23.993229 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:24.013079 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] MFA Rules not processed for user `9e5f6c1a1f07409483d732db05256f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:50:24.014320 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:24.022092 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1446/2883] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:50:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:50:26.874665 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:26.877709 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:26.901912 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b001c-3523-41cb-ac58-9f2e05da394b service cinder] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:26.906468 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:26.907302 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:26.907589 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:26.908404 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:26.910244 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:26.996489 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:26.998016 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:27.020107 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1438/2884] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:50:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:50:27.976526 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:27.981949 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:27.990195 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e6d3c-018d-494c-ad97-5a50952b5953 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:27.994441 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:27.995324 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:27.995840 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:27.996159 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:27.998223 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:28.012684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:28.014259 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:28.025837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1447/2885] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:50:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:50:37.017963 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=93 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:38.020961 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=5 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:44.215260 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:44.225495 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece02e4-9402-4a26-b310-00b655da241d service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:44.230001 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:44.230656 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:44.230928 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:44.231286 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:44.233272 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:44.244373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:44.245797 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:44.254644 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1439/2886] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:50:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:50:46.734085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:46.743402 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bac741-8951-4d11-a1b1-261397760968 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:46.747300 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:46.747928 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:46.748210 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:46.748482 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:46.750462 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:46.762850 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:46.764289 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:46.773832 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1448/2887] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:50:53.258992 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:53.260722 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:53.276544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e311b1-4ea7-468a-9bca-bc527cf38d5a service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:53.281276 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:53.281918 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:53.282240 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:53.282520 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] PATH_INFO: `/v3/users/90fa94f90d49485a81d502929bcd0b93` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:53.283711 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90fa94f90d49485a81d502929bcd0b93)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:53.285673 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:53.338542 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90fa94f90d49485a81d502929bcd0b93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.339214 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90fa94f90d49485a81d502929bcd0b93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.347776 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90fa94f90d49485a81d502929bcd0b93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.355394 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90fa94f90d49485a81d502929bcd0b93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.368066 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90fa94f90d49485a81d502929bcd0b93'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.374632 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1440/2888] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:50:53 2022] DELETE /identity/v3/users/90fa94f90d49485a81d502929bcd0b93 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:50:53.569600 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:53.580282 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5973f-bb9b-4eab-aa8f-d9b3c43e3b46 service neutron] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:53.637876 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:53.638549 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:53.638828 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:53.639102 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] PATH_INFO: `/v3/projects/9aadaf242f674436a45eae700fb52fc1` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:53.640245 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aadaf242f674436a45eae700fb52fc1)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:53.642244 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:53.689725 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aadaf242f674436a45eae700fb52fc1', 'request_id': 'req-07f6807e-60fa-4381-8e63-af963a9507d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.697890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aadaf242f674436a45eae700fb52fc1', 'request_id': 'req-07f6807e-60fa-4381-8e63-af963a9507d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.704469 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aadaf242f674436a45eae700fb52fc1', 'request_id': 'req-07f6807e-60fa-4381-8e63-af963a9507d1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.715058 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] The token cache is being invalidate because project 9aadaf242f674436a45eae700fb52fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:50:53.715355 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.716012 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.717535 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1449/2889] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:50:53 2022] DELETE /identity/v3/projects/9aadaf242f674436a45eae700fb52fc1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:50:53.721698 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:53.723332 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:53.737258 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74d6db-f66e-4a34-96e7-b630e1501993 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:53.796300 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:53.796926 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:53.797206 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:53.797476 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] PATH_INFO: `/v3/users/e6a58dddb41a477fb1552e9a0deb01b6` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:53.798666 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a58dddb41a477fb1552e9a0deb01b6)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:53.800580 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:53.848378 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a58dddb41a477fb1552e9a0deb01b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.849086 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a58dddb41a477fb1552e9a0deb01b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.857544 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a58dddb41a477fb1552e9a0deb01b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.864010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:53.866274 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a58dddb41a477fb1552e9a0deb01b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.887020 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a58dddb41a477fb1552e9a0deb01b6'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:53.893631 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1441/2890] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:50:53 2022] DELETE /identity/v3/users/e6a58dddb41a477fb1552e9a0deb01b6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:50:54.072661 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:50:54.074121 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:50:54.092085 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6807e-60fa-4381-8e63-af963a9507d1 admin admin] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:50:54.149036 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:50:54.149701 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:50:54.149983 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:50:54.150287 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] PATH_INFO: `/v3/projects/8f7fa836383e4ffaa6209e73ed6d946a` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:50:54.151409 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f7fa836383e4ffaa6209e73ed6d946a)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:50:54.153330 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:50:54.173790 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:50:54.202305 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f7fa836383e4ffaa6209e73ed6d946a', 'request_id': 'req-61e96e6b-c9b7-43df-b77a-11e39b461cc1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:54.211684 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f7fa836383e4ffaa6209e73ed6d946a', 'request_id': 'req-61e96e6b-c9b7-43df-b77a-11e39b461cc1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:54.218271 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f7fa836383e4ffaa6209e73ed6d946a', 'request_id': 'req-61e96e6b-c9b7-43df-b77a-11e39b461cc1'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:54.228802 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] The token cache is being invalidate because project 8f7fa836383e4ffaa6209e73ed6d946a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:50:54.229109 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:54.229731 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-61e96e6b-c9b7-43df-b77a-11e39b461cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:50:54.231272 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1450/2891] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:50:54 2022] DELETE /identity/v3/projects/8f7fa836383e4ffaa6209e73ed6d946a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 02:51:03.889983 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:51:04.225953 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:51:22.047433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:51:22.049222 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:22.130032 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265679b-ca11-4392-9469-f76a548feac7 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:51:22.209430 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:51:22.210061 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:51:22.210373 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:51:22.210652 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:51:22.212418 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:22.224420 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:51:22.226010 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:51:22.237231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1442/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:51:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:51:31.155704 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] REQUEST_METHOD: `POST` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:51:31.156277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:51:31.156277 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:51:31.175937 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.auth.core [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] MFA Rules not processed for user `9e5f6c1a1f07409483d732db05256f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 02:51:31.177164 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:31.238249 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1451/2893] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 9 02:51:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 02:51:31.483352 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:51:31.486231 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:31.550237 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f99a87-eea0-4f2f-90f9-126d670094b0 service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:51:31.554609 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a425b3b4b3f34217ba148c2f855e0068', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:51:31.555216 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:51:31.555493 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:51:31.555764 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:51:31.557984 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:31.568531 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:51:31.569955 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:51:31.578499 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1443/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:51:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:51:32.161044 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:51:32.162576 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:32.243086 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62ebbc-de6e-4182-9635-2737ecd25c41 None None] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:51:32.248524 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:51:32.249139 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:51:32.249420 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:51:32.249698 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:51:32.251598 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:32.263008 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:51:32.264605 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:51:32.274618 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1452/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:51:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:51:40.507433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:51:40.509156 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:40.526048 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0a542-7d03-4d88-a024-59b261d206c8 service neutron] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:51:40.532254 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad4a7ffff8f842969d782603cbb8fb89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:51:40.532969 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:51:40.533315 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:51:40.533675 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:51:40.535526 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:51:40.548433 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:51:40.550065 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:51:40.560813 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1444/2896] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:51:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:51:42.270481 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=57 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:51:50.556528 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=94 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:52:27.516191 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:52:27.526858 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9a768-8de2-4bb5-86e7-ce9a107fdb17 service placement] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:52:27.532395 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:52:27.533100 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] REQUEST_METHOD: `GET` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:52:27.533572 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:52:27.534047 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:52:27.536673 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.fernet_utils [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:52:27.551491 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:52:27.554384 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:52:27.570490 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1453/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:52:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:52:27.581965 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:52:27.583865 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:52:27.651502 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c2692-e430-4574-9a8e-2ec5dff42fce service nova] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:52:27.656117 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:52:27.656745 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:52:27.657026 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:52:27.657321 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] PATH_INFO: `/v3/users/9e5f6c1a1f07409483d732db05256f1b` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:52:27.658439 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5f6c1a1f07409483d732db05256f1b)` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:52:27.660350 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:52:27.707407 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5f6c1a1f07409483d732db05256f1b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:27.708152 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5f6c1a1f07409483d732db05256f1b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:27.716537 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5f6c1a1f07409483d732db05256f1b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:27.723629 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5f6c1a1f07409483d732db05256f1b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:27.736751 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.notifications [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5f6c1a1f07409483d732db05256f1b'} {{(pid=80496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:27.743198 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1445/2898] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 9 02:52:27 2022] DELETE /identity/v3/users/9e5f6c1a1f07409483d732db05256f1b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:52:27.955918 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] Authenticating user token {{(pid=80497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:52:27.970045 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f23de-22fe-4416-a41d-4eee737f126f service placement] Validating token access rules against request {{(pid=80497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:52:28.029264 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1cb94a6532045fc84cecdb848b352fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aaeb91952f2c4c9f8cf8e3a10b5518c9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:52:28.029890 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:52:28.030190 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:52:28.030468 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] PATH_INFO: `/v3/projects/d488e2c8749640a0b3a9b0037051ff12` {{(pid=80497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:52:28.031711 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d488e2c8749640a0b3a9b0037051ff12)` {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:52:28.033620 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] RBAC: Authorization granted {{(pid=80497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:52:28.081837 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd488e2c8749640a0b3a9b0037051ff12', 'request_id': 'req-885d4a4a-f89b-4e97-a257-386c2da0b520'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:28.091075 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd488e2c8749640a0b3a9b0037051ff12', 'request_id': 'req-885d4a4a-f89b-4e97-a257-386c2da0b520'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:28.097816 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd488e2c8749640a0b3a9b0037051ff12', 'request_id': 'req-885d4a4a-f89b-4e97-a257-386c2da0b520'} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:28.109378 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] The token cache is being invalidate because project d488e2c8749640a0b3a9b0037051ff12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 02:52:28.109670 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:28.110426 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG keystone.notifications [None req-885d4a4a-f89b-4e97-a257-386c2da0b520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 02:52:28.111958 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: [pid: 80497|app: 0|req: 1454/2899] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 9 02:52:27 2022] DELETE /identity/v3/projects/d488e2c8749640a0b3a9b0037051ff12 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 02:52:37.739605 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:52:38.105939 dv-cinder-852461-1-iscsi devstack@keystone.service[80497]: DEBUG dbcounter [-] [80497] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 02:53:47.474980 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Authenticating user token {{(pid=80496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 02:53:47.476573 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:53:47.545663 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf8535-8796-486b-8cd3-740144456f56 admin admin] Validating token access rules against request {{(pid=80496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 02:53:47.609653 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dea38c5b41e5432d9f94f3248824e262', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9deddd84460e4ee9af2f645e34eda375', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 02:53:47.610296 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] REQUEST_METHOD: `GET` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 02:53:47.610561 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] SCRIPT_NAME: `/identity` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 02:53:47.610831 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 02:53:47.612744 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.fernet_utils [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 02:53:47.625730 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 02:53:47.628141 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2d43-c3e1-4b1a-806d-ee1a2c435a61 service placement] RBAC: Authorization granted {{(pid=80496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 02:53:47.642683 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: [pid: 80496|app: 0|req: 1446/2900] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 9 02:53:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 02:53:57.636256 dv-cinder-852461-1-iscsi devstack@keystone.service[80496]: DEBUG dbcounter [-] [80496] Writing DB stats keystone:SELECT=58 {{(pid=80496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}